Introduction

While less than 4% of the world’s population are international migrants, the experience of international migration has risen by 54% globally between 1990 and 2020 (McAuliffe & Triandafyllidou, 2021). Further, the number of people forcibly displaced due to persecution, conflict, and violence is on the rise, reaching a high of over 110 million by mid-2023 (UNHCR, 2023), an unprecedented proportion of which are children. For all migrants, racism, discrimination, and pressure to assimilate (and thereby abandon links to their country and culture of origin) are often noted to accompany dislocation (Fenta et al., 2004). However, refugee youth are at increased risk of exposure to potentially traumatic events before, during, and after migration which raises concerns for their welfare (Lustig et al., 2004). Exposure to trauma is well-established predictor of adverse mental health outcomes internationally (Zarse et al., 2019). For refugee youth specifically, systematic reviews have shown they experience higher rates of posttraumatic stress disorder (PTSD), depressive symptoms, and emotional and behavioural problems compared to youth of the general public (e.g., Bronstein & Montgomery, 2011). These findings indicate an urgent need for effective interventions for refugee youth.

Despite the potential traumas of migration, forcibly displaced children also demonstrate resilience (Lustig et al., 2004), or positive ways that children adapt and function considering their developmental level and trauma histories (Jafari et al., 2022). Previous literature has shown the potential for connection to community, family, friends, education, and culture to support the resiliency of refugee youth (Pieloch et al., 2016). Schools (when supportive) can become the scaffolds for these children to rebuild their sense of self and relationships and to mitigate the effects of trauma-exposure on their mental health (Guruge & Butt, 2015; Herati & Meyer, 2023).

Additional research has focused on the role of schools in mitigating the effects of trauma (Herati & Meyer, 2023) and as effective settings for interventions for refugee children (Guruge & Butt, 2015). Interventions that combine these lines of research by the use of multi-pronged programming, in which all students in a given setting are provided with support and students with specific needs are provided additional support, have been shown to have positive effects on refugee youths’ wellbeing (Ellis et al., 2013; see Sullivan & Simonson, 2016, for review). Cumulatively, these findings point to the potential benefits of multi-tiered interventions and the use schools or other informal settings as suitable environments for the delivery of resilience and wellbeing interventions among refugee youth.

Despite the evidence of the need for mental health and educational interventions and schools being identified as potentially successful sites for interventions, addressing these needs among refugee youth remains challenging. Partially, this challenge arises from the complexity of largescale immigration, where, and as Weine (2011) outlines, interventions are often “trying to address multiple types of potentially negative outcomes, in multiple culturally distinct refugee groups, across multiple systems that are often not well coordinated, in ways that vary across time, with persons living in or near poverty, in low-resource environments” (p. 416). Coupled with this complexity, the application of Western models of mental health to refugee populations risks implementing interventions that are unacceptable, irrelevant, or inappropriate for specific groups, and lead to the underutilisation of mental health supports (Lustig et al., 2004). To adapt interventions to forcibly displaced youth, Harker Roa and colleagues (2023) highlight the need to balance fidelity and flexibility within interventions. Specifically, they argue for an explicit adaptation process defining elements of the intervention that can be adapted, need to be adapted, and should never be adapted to local context. This need for cultural adaptation is reflected across several different health (including mental health) interventions for forcibly displaced youth (Perera et al., 2020; Singh et al., 2021).

One model for developing culturally appropriate interventions that also accounts for contextual complexity is ‘co-creation’. The co-creation process works to increase community buy-in and build trust by directly involving the intervention target (i.e., refugees) in the design of the intervention (Afifi et al., 2011). In reviews of the health literature on refugee communities, interventions that involve the community in a co-creative manner are those that account for the diversity that exists within refugee populations (Filler et al., 2021) to increase the acceptability and impact of the interventions designed for refugee youth (Frounfelker et al., 2020). Additionally, the engagement with the co-creation process has been shown to benefit refugees as the process allows them to be “heard and seen” (Simonsen & Ryom, 2023, p. 398). Therefore, co-creation may benefit refugee communities by improving both the processes and outcomes of interventions.

While co-creative strategies show potential benefits to target populations, including refugees, implementing co-creation remains a challenge. For example, expanding the use of co-creation may be limited by a lack of common language, inconsistent frameworks, and poor understanding of the mechanisms of co-creation (Halvorsrud et al., 2021). Further, co-creation may require shifts in perspective (Abdi et al., 2022) and the incorporation of approaches that are specific to co-creation (e.g., allowing time for participants to emotionally process their experience while taking part in the intervention; Brown & Choi, 2018). Process evaluations, which explain the mechanisms and contexts of interventions to understand the roots of their success or failure (Grant et al., 2020), may thus help to address prevailing challenges within co-creation.

Previous process evaluations of co-creation with refugee communities have demonstrated their potential benefit. By using a variety of data, process evaluations have been used to elucidate both the engagement with and quality of co-created interventions delivered for refugee youth (Bleile et al., 2021). Further, process evaluations have led to suggested improvements to co-created interventions. For example, a process evaluation of the Qaderoon project, a mental health intervention guided by a community coalition, identified barriers and facilitators to the intervention’s implementation that allowed the project team to make intermittent changes to increase the success of the intervention (Nakkash et al., 2012). By analysing how co-created interventions are implemented and how they can be improved, process evaluations can thus provide information to support the scaling up of co-created interventions going forward. Specifically - and consistent with recent recommendations from the British Medical Research Council (BMRC) emphasising the role of process evaluations to identify barriers and facilitators to intervention implementation to better understand the mechanisms of interventions and generate recommendations to better inform policy and practice (Moore et al., 2015) - process evaluations may be particularly useful to identify, and subsequently address, some of the existing challenges to implementing and scaling co-created interventions for refugee youth. The aim of the current study was therefore to apply process evaluation techniques within the co-created needs assessment stage of the REFUGE-ED project in order to identify barriers, facilitators, and recommendations for co-creating health interventions with refugee youth.

The REFUGE-ED Project

REFUGE-ED is a Horizon 2020 European Union (EU) funded project that aimed to co-create educational and mental health and psychosocial support (MHPSS) interventions to support the social inclusion of refugee and migrant youth between the years 2021 and 2023. The project consortium was comprised experts from nine organisations across seven EU countries; namely Autonomous University of Barcelona, University of Florence, New Bulgarian University, KMOP, Trinity College Dublin, The MHPSS Collaborative, Save the Children Sweden, The International Federation of Red Cross and Red Crescent Reference Centre for Psychosocial Support (PSC), Support Group Network, and Cooperazione Internazionale Sud Sud (CISS). Consortium members worked with 46 implementation sites in six EU countries: Bulgaria, Greece, Ireland, Italy, Spain, and Sweden. Aligned with the EU recommendation of including multiple stakeholders in supporting social inclusion of refugee youth (European Union, European Commission, 2020), a core component of REFUGE-ED was the dialogical co-creation process. Outlined in further detail elsewhere (see Sordé-Martí et al., 2023) key to dialogical co-creation is the relationship between various stakeholders, including end-users of the research (i.e., refugee children and families), and researchers. This relationship is conceptualised as bi-directional, wherein stakeholders guide the research process based on their lived experiences and researchers guide the research process based on their scientific expertise.

The Current Study

While co-creation within REFUGE-ED was outlined as a guiding principle throughout the project, the current paper will evaluate the process of co-creation as an intervention that preceded the implementation of educational and MHPSS programming. This phase included conducting interviews/focus groups with stakeholders asking them about their needs, synthesising these findings, reporting the needs back to stakeholders, needs prioritisation, providing stakeholders with a description of potential intervention strategies based on their needs, and engaging stakeholders in selecting an evidence-based intervention to address these needs. The focus in this paper is on the perspectives of those implementing the co-creation process, be it members of the REFUGE-ED consortium or those working in sites where various educational or MHPSS programmes would eventually be rolled out. The perspectives of children and adolescents (both from host communities and refugee and asylum seeking-youth) were sought as part of the co-creation process (e.g. needs identification and prioritization, selection of evidence-based interventions); however their perspectives on the co-creation process itself were not collected as part of this process evaluation. The authors acknowledge this a limitation to the current study.

Method

Ethics

Ethical approval for the data collection included in the current paper was provided by University of Dublin, Trinity College. All participants provided written or oral consent for their participation in the research, although their participation within the programmes provided as part of the REFUGE-ED project was not dependent on their participation in this specific data collection process. Data were stored according to procedures approved by the University of Dublin, Trinity College Data Protection Officer to ensure the confidentiality of participant information.

Participants and Procedures

In order to capture implementors’ experience with the co-creation process, participants were members of the REFUGE-ED consortium (N = 42) and staff of three implementing educational sites (N = 16). Co-creation data specific to the needs assessment phase of REFUGE-ED was gathered via a combination of online and in-person methods and included gathering, audio-recording, and transcribing the following relevant data sources: (1) ‘consortium meetings’ and ‘needs analysis meetings’ taking place between September 2021 and June 2022 (coinciding with the needs assessment phase of REFUGE-ED); (2) consortium member online focus groups, interviews, and surveys; and (3) in person focus groups and key informant interviews with implementation site staff held in Ireland and Bulgaria.

Of the consortium meetings that took place during the needs assessment phase, seven were recorded and transcribed using automatic transcription software via Microsoft Teams, which were then reviewed against the recording for accuracy. One additional consortium meeting was attended in person by consortium members in March 2022 and was not recorded. The meeting minutes were therefore included in the analysis. Next, ‘needs analysis meetings’ consisted of meetings held amongst consortium staff members via Microsoft Teams, where each country presented the results of the needs analysis within their country and team members discussed facilitators of and barriers to the co-creation process. Automatic digital transcripts of meeting recordings were generated via Microsoft Teams. All transcripts were reviewed twice to identify passages that reflected on the needs analysis process. The identified passages were then checked for accuracy against the recording and compiled. Compiled passages were then reviewed to ensure that they contained information relevant to the process evaluation. All implementing countries and consortium organisations were present in the transcribed meetings including Spain (n = 6), Italy (n = 6), Greece (n = 3), Bulgaria (n = 6), Sweden (n = 8), Ireland (n = 5), and the two mental health and psychosocial support (MHPSS) implementation teams (n = 8).

Two consortium members declined to take part in the process evaluation at the needs analysis phase. All consortium members who consented to the evaluation were given the option to take part via a one-on-one interview, a focus group with other consortium members from their country, or an anonymous online survey to reflect on the needs analysis and co-creation process. The questions in all three options asked consortium members to identify what methods were used in the needs analysis and co-creation phases, how suitable they thought these methods were for children, and to reflect on the strengths, barriers, and recommended changes to these specific phases. Probing questions for further detail and elaboration followed these questions based on participants’ responses. The interview guide for these focus groups and online survey can be found in Online Resource 1. Consortium members from the Bulgarian (n = 2), Spanish (n = 3), Irish (n = 2), and Swedish (n = 4) teams agreed to take part in focus groups or interviews. Consortium members from the Bulgarian (n = 1), Swedish (n = 2), Italian (n = 5), and Greek (n = 1) teams completed the online survey in addition to one participant who did not identify their organisation. The full transcripts of each interview and focus group as well as all survey responses were included in the analysis.

Finally, a series of interviews and focus group discussions (FGDs) were held by consortium members with implementation site staff involved in the process of co-creation in February 2023. The FGDs were aimed at evaluating stakeholders’ perception of the project thus far and for stakeholders to provide their perspective on strengths and challenges of the project. As a part of these interviews, some stakeholders were asked about the co-creation process. The interview guide for these focus groups can be found in Online Resource 2. Probing questions for further elaboration followed questions in the interview guide. Selective transcripts of the interviews were then prepared by each consortium team for the stakeholders in their country. In the current analysis, selective transcripts were reviewed to identify interviews and focus groups where the needs analysis and co-creation steps were explicitly discussed. Of the interviews/focus groups conducted, consortium members from Ireland conducted one focus group with two staff members and two interviews with staff from one implementation site. Consortium members from Bulgaria conducted two focus groups with five (4 female, 1 male; all country of origin Bulgaria) and seven (6 female, 1 male; all country of origin Bulgari) staff members from two implementation sites. Table 1 summarises the data sources reviewed, the number of data points, number of participants, and time of data collection for each included data source.

Table 1 Data Sources for Analysis
Full size table

Analysis

A qualitative content analysis using both deductive and inductive approaches was applied (Elo & Kyngäs, 2008). First, an initial familiarisation phase was conducted during which the data was collected, transcribed, and reviewed repeatedly. During this phase, all data that were relevant to the needs analysis phase of the project was identified, compiled, and uploaded to NVivo (Version 12 Plus) for analysis. Next data were re-read in the context of the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2022). Designed in response to a lack of comparability and consistency for the identification of implementation facilitators and barriers across the literature (Damschroder et al., 2009), CFIR was chosen given its recognition as a generalisable approach to evaluating the process of implementation. Recently updated, the current CFIR framework consists of 83 constructs and subconstructs across eight domains that encapsulate a broad range of program characteristics (Damschroder et al., 2022). During this second phase, the first author reviewed CFIR constructs that were described in the data. The first author presented frequently identified constructs to the research team resulting in the following four constructs being identified: teaming, tailoring strategies, engaging innovation recipients, and doing. Operational definitions of these constructs, adapted to the REFUGE-ED project were then developed and are summarised in Table 2.

Table 2 CFIR Constructs Project Definitions
Full size table

Similar to previous intervention process evaluations (Damschroder & Lowery, 2013; Liang et al., 2015; Varsi et al., 2015) deductive analysis was thus used to align each data point to the relevant CFIR construct. These constructs were then also divided into subordinate categories of barriers and facilitators (i.e., processes which supported or interfered with co-creation). For example, during a meeting a consortium member stated: “We are organising more personal meetings with the selective participants that we think that can be involved in the project: we are meeting them for a coffee to strengthen the relationship of trust because it is a very important element as we request them to get involved in the actions.” This was coded as a facilitator to Engaging Innovation Recipients as it spoke to a strategy for consortium members to motivate implementation site staff to participate in the project.

An inductive analysis was then conducted via the creation of semantic codes summarising each data point. The example quote above was coded as “meeting with participants.” A latent analysis of these codes was then conducted via agreement between three co-authors to generate superordinate themes, whereby codes were revised until the team unanimously agreed on the revised themes prior to completing the analysis. The latent codes were then reorganised into the appropriate theme. The aforementioned example quote was assigned to the theme ‘Building Relationships’ within ‘Engaging Innovation Recipients.’

Transcripts were then revised again according to these themes to ensure that the data points were coded consistently to the themes’ definitions. Any changes to the themes as a result of this process were discussed and agreed upon by four of the co-authors.

Results

Eight facilitators and eight barriers were identified across the four CFIR constructs. A summary of the facilitators and barriers identified in each phase is provided in Table 3 below.

Table 3 Facilitators and Barriers Identified
Full size table

Teaming Facilitators

Shared Site Needs and Barriers

Although contexts for co-creation differed greatly across consortium partners (such as their national context [e.g., cultural and political climate] and local setting [e.g., in a school, residential centre, or community centre with participants from diverse migrant backgrounds]), the needs of youth refugees identified through co-creation were often shared across the consortium. Consortium members identified how valuable the shared consortium space (i.e., consortium meetings and opportunities to exchange ideas) was to collaborate for the planning and implementation of co-creation (e.g., developing an analytic framework for data collected from pilot sites). This view that co-creation happened within the international consortium, as opposed to each team operating as an individual, increased feelings of self-efficacy and resourced members to develop concrete co-creative steps to guide educational as well as MHPSS programming:

Well, I think that this is also very useful and I think that the challenges that, and the needs that have been presenting to today, are very much shared by many sites as you will see in the Spanish case. So I think that this is good, you know because it will allow us to, to, to intervene with successful results.

Team Member Expertise

The range and depth of consortium expertise allowed for consortium members to collaborate on the co-creation of MHPSS and educational interventions. As the individuals and organisations each brought expertise to the project, the consortium represented a team which was greater than the sum of its parts:

Consortium Member 1: “So something I was thinking is that even if we go for the identifying and promoting the MHPSS component of the successful educational actions, I think that it might be needed like some general training on MHPSS, right? …”

Consortium Member 2: “Yeah. And this is what, this is what we do in the PS Centre on a daily basis. Like we have a ton of training packages that are, not all in the languages that you have, might not have it in Swedish, but we have an English, French a bunch of different languages, and we make them so you can adapt them to your own context…”

Teaming was thus supported by this wide breadth of knowledge as diverse expertise existed within consortium spaces.

Teaming Barriers

Lack of Conceptual Unity

A lack of conceptual unity around core project principles and concepts were raised as challenges to co-creation within the consortium. Specifically, some consortium members expressed a lack of clarity regarding the project’s intended locus of action and theory of change. One consortium member noted that this confusion may reflect the field of co-creation:

And there’s a real gap in terms of how can you actually design participatory theories of change because they tend to be quite an academic languagey type thing. So it could be really interesting to try and develop a methodology to try and do that with children.

Consortium members also discussed divergences in their conceptualisation of co-creation. For example, within consortium meetings centred on co-creation the concept of co-adaptation, that is improving existing interventions, as it relates or diverges from co-creation, was discussed:

But basically, our main idea right now is, and the one thing I haven’t talked about, I will talk about in a second, is one, how can we improve, … how can we improve the successful education actions as they are? How can we strengthen the MHPSS? And maybe that’s more about co-adapting than cocreating these SEAs.

Varying levels of experience with co-creation, lack of accessible definitions of co-creation within the field, and differing conceptualisations of the role of co-creation in SEAs and MHPSS interventions may have limited Teaming within the consortium. Consortium members expressed the need for greater conceptual clarity as a collective before working with the community.

Lack of Practical Planning

Related to varying conceptualisations of project principles, a lack of planning and clear steps for the process of co-creation challenged collaboration. This includes concerns raised by consortium members regarding feasibility, capacity, time, coordination between work packages, and resources required to complete the steps of co-creation. These concerns directly hampered consortium members’ contributions to each team. For example, concerns about receiving ethical approval in a timely manner to begin co-creation led one consortium team to submit an ethics application before they could receive the interview template that a different consortium team was creating. Thus, this concern for time due to limited practical planning of the time requirements for ethics processes and developing protocols, prevented collaboration at this point. After receiving ethical approval and the protocols, the team was able to update their interview guide. However, this process created a challenge that could have been prevented through planning.

A further challenge to teaming was the limited understanding of the practical steps of co-creation. While part of the project framing was defined steps to co-creation, consortium members reported varying understanding of how these steps were to be executed. For example, the MHPSS consortium members expressed the challenge of collaborating with consortium teams to co-create MHPSS interventions when broad needs were being reported from sites during the needs analysis. As consortium members were facing this barrier, a ‘Co-Creation Roadmap Meeting’ was held in June 2022 to operationalise the co-creation process. Within this meeting, disagreement was voiced regarding the steps to co-creation. An exchange from this meeting below demonstrates the lack of agreement regarding the practicalities of co-creation:

Consortium member 1: “Yes it’s just like I I want to make sure that all of us are on the same path that, OK, the the seven steps that we defined for the dialogical creation process, like this is something that we already agreed on, we already discussed.”

Consortium member 2: “I think the problem with that is that we haven’t figured out how to actually co-create. So I think how to prepare people for co-creative process, how to ask them what their needs are, I think it’s different. We haven’t found a way to assess the MHPSS needs yet, this is one issue that we’re dealing with, how to assess not only needs in general but specific how to address mental health and psychosocial support needs. And then to utilize that needs assessment to integrate it into what you have already established and well founded on what are ways to integrate successful educational actions. So I think those two steps are least clear of all.”

Tailoring Strategies Facilitators

Flexibility

The tailoring of co-creation to implementation sites was facilitated by the flexibility of the protocols, the researcher, and within the relationship between the two. Consortium wide research protocols, such as focus group discussion guides, were described as adaptable to context and stakeholders. Consortium members reflected on using templates with open-ended questions as a way to balance to need to standardise elements of co-creation while allowing the process to be adapted to sites:

So there has to be like a balance on having all the information as systematized as possible so as comparable as, as possible in between site but also as the sites are, like have like particularities you can, you want also like conserve this this particularity. So I think it’s difficult, this balance. And I think it was for me it was, it was good to have like better like kind of open templates because that meant that I could reflect more literal, kind of, the reality of each site because it was by different between different reception centres and also for example, primary schools.

Researchers were also seen as flexible to site needs, adjusting their co-creation plans based on sites. The flexibility of the researcher and protocols were interconnected as the flexibility in both facilitated adaptation:

FGD Facilitator: “… in terms of the child friendliness of the protocols, what did you kind of think of them? Did you feel they were not child friendly at all, somewhat child friendly but room for improvement or completely child friendly?”

Consortium member: “That’s a very good question. I think that because we were not reading specifically the protocol, that helped. (FGD Facilitator: OK) But I’m sure that if I read out loud the question to the children and do not explain, I don’t think that that could work, you know.”

Allies to Connect Researchers to Sites

‘Allies’ were identified as key stakeholders who supported the adaptation of co-creation to sites through their relationship to consortium members. Specifically, allies supported the adaptation by providing site access and context. Reflecting on the support of allies, one consortium member stated: “they know how this these systems work and much more are, are, are much more let’s say on the ground. So they are always ready to share these type of things.” Additionally, consortium members’ relationships within their team allowed knowledge to be shared to support adaptation. These relationships and ‘networks’ supported the flow of information and increased consortium members’ understanding of context and sites’ needs.

Tailoring Strategies Barriers

System Rigidity

While some participants identified flexibility as a facilitator to adaptation, others referenced points of rigidity seen across systems, including research protocols, research ethics, government, and intervention methods. Consortium members highlighted that research protocols did not consider the stakeholders’ role and context, thus creating barriers to adaptation. In reference to this rigidity, one stakeholder shared:

And I recognize that it [focus group protocol] was used as a frame that you could jump off from, but I, I don’t remember us finding kind of, moving from that quite like abstract or large into the specific like relevant for, for our groups.

Consortium members’ ability to adapt steps to co-creation may also have been limited by bureaucratic delays accessing the implementation sites (e.g., some of the implementation sites agreed to participate in REFUGE-ED, however these agreements were superseded, delayed, and sometimes blocked by governmental policies and restrictions). Further delays were caused by lengthy ethics committee processes. Once ethical approval was received, changes to co-creation processes to tailor strategies to sites was limited by some of the requirements of the ethics processes:

What was unfortunate at one point of the co-creation anyway, was that, at that one of the parents said ‘wouldn’t a suggestion box be great at the front of the school and people could just anonymously put things, put things in.’ But at this point that they said this to me, we were already gathering this information and needed to gather it and make sense of it to present it for the next step of the project. And so it wasn’t possible to do it within the time frame. And yet that was something that would have been a really co-created and means of data collection.

Both the limited time which diverged from expectations of the grant agreement, as well as the scope for tailoring strategies a posteriori, hindered the adaptation of co-creation to implementation sites’ needs.

Research as Non-Priority

Consortium members identified competing priorities within implementation sites as a barrier to tailoring strategies. Implementation sites expressed that they had limited capacity to engage with the research or provided limited access to the site due to other activities. Other stakeholders also expressed prioritisation of objectives beyond the project’s aim, most often increasing the fluency of the refugee and migrant youth in the site’s native language. These sentiments indicate a lack of prioritisation in the process of co-creation within the current research project. Consortium members reported that competing priorities and demands were connected to implementation site drop-out and limited or lack of access to participating implementation sites:

…School administration in the municipality level are denying to take part because they say ‘we are busy and maybe you can go ahead somehow but we don’t like to take part in the project.’ So this is a bit challenging for us and still we couldn’t convince them.

The limitations of co-creation not being prioritised presented challenges for understanding sites and tailoring strategies to their needs.

Engaging Innovation Recipients Facilitators

Building Relationships

Relationships between stakeholders and consortium members were facilitated by stakeholders’ understanding of the project, hope that the project could create a needed change, and feeling of connection to the project as a whole. Consortium members discussed concrete steps taken to build these relationships with stakeholders including regular meetings, sharing information, and making stakeholders feel heard. A site staff member from Bulgaria noted the “interactive approach” of the project and indicated the perceived value of co-creation: “the co-designed and co-implemented approach is the right one. It is slower in time but more sustainable.”

Engaging Innovation Recipients Barriers

Not Meeting Participants Where They Are at

Contrary to building relationships with stakeholders, engaging stakeholders was challenged by limited trust-building in combination with a lack of understanding of sites’ context and of the challenges co-creation can entail. At times, consortium members were seen as ‘outsiders’ to sites which may have been exacerbated by members not taking time to build trusting relationships with stakeholders. As a result of being separate from the site, consortium members may not have understood or accounted for how factors such as power dynamics between stakeholders, scepticism about research, differences in language spoken, uncertainty regarding the future, and high demands created barriers to stakeholders engaging with co-creation. Consortium members also may not have accounted for the challenge in getting buy-in for the process of co-creation, as explained by this site leader:

To get people involved, they need to know if I sign up to this, essentially what am I going to be doing and how much training is involved? And how much time in my class is going to take up? And who’s gonna be coming into my class and that kind of thing? Those things, we didn’t have them.

Consortium members may have not met participants where they were at by failing to address the practical challenges that co-creation posed for gaining site buy-in.

Doing Facilitators

Consortium for Collaboration

Access to a wider consortium for collaboration supported consortium members in carrying out the steps of co-creation outlined in the project. Consortium members expressed the support that resources created within the consortium, such as interview templates and analytical frameworks, provided in the implementation. Additionally, consortium members expressed that the international presentations to sites and the internal consortium meetings supported stakeholders and the consortium in understanding the process, respectively.

Researcher’s Skill

Multiple research skills and expertise were identified within the data that facilitated co-creation. These skills included language skills relating to both type of language and style of language that allowed them to engage with stakeholders. More broadly, consortium members had expertise and skills in facilitating conversations and focus groups with stakeholders regarding sensitive topics. This included the skill to perceive stakeholder’s emotional responses, connect with stakeholders, validate experiences, adapt to unplanned topics, and lead focus groups in a non-directive manner.

On multiple occasions, consortium members were also able to pre-emptively identify possible points of conflict in the methodology and propose alternative solutions that allowed for co-creation to continue. For example, consortium partners were tasked with hosting assemblies at each site to present the needs back to the implementation sites and facilitate prioritisation of needs. When planning this step, one consortium partner decided to host the assembly with multiple sites at a time. They explained this decision as a way of increasing the efficiency of the process and as a way of supporting the implementation sites:

I’m thinking, for example in the case of [implementing site], can we, [a combined assembly] can be a way for school to emulate, to see each other and see ‘this school does so maybe we can also try’ and maybe can be also a way to foster all schools to, to, to start with this process.

The expertise consortium members brought to the research process supported multiple steps of the co-creation process.

Trust and Relationship

A variety of relationships were identified as facilitators of co-creation including between consortium teams and implementation site leadership, consortium members and focus group participants, site staff and focus group members, focus group members, and consortium members within a team. Some of these relationships existed before the project’s implementation and consortium members identified these existing relationships as facilitators of co-creation. For example, consortium members spoke of grouping participants with existing relationships in focus groups as a way of capitalising on “natural teams” to support implementation. Consortium members also discussed strategies for establishing trusting relationships through common experiences (for example, migration experiences), communication, and clear project values. A project value emphasised by one consortium team was the importance of stakeholders being held as the experts of their community with a shared ownership in the co-creation process. Reflecting on this, a consortium member shared:

When families realise that this project, or this interview, is around the cocreation, so we want to co-create solutions, we don’t have the answers, we don’t know what to do, so we want to co-create this together. I think here’s where things turn, you know. They felt more, they the, the, there is more ownership and there’s more, you know, in in in the process.

Doing Barriers

Differing Priorities of Stakeholders

In addition to differing priorities being a barrier to tailoring strategies, it also presented as a barrier to implementing co-creation. Consortium members reported the challenge presented when stakeholders either minimised or disagreed on the priorities identified for the project. Specifically, stakeholders reported that assimilation and language fluency were key needs and either disagreed that mental health was a concern or did not see it as a priority. Speaking to this, one consortium member stated:

When we presented back the needs, no one voted for needing MHPSS training. They are very preoccupied with language barriers and needs for supporting language learning. We think they need some basic training to be able to recognize that these needs exist, but they aren’t interested in this. It’s like an emergency situation, we’re just focused on food water etc. MHPSS gets pushed back as not seen as a priority.

This difference in prioritisation acted as a barrier to co-creating interventions with stakeholders.

Lack of Buy-In to the Process

A lack of buy-in to the process of co-creation limited stakeholders’ participation and worked against the principle of co-creation of whole community involvement. For example, some consortium members reported that only certain sub-groups of stakeholders took part (e.g., migrant staff or recently immigrated families). In some instances, this was a result of sites restricting the access of consortium members. Consortium members also identified other possible causes of this including concerns that the intervention being co-created was too abstract, a lack of clarity on what was being co-created, and a concern that the co-created intervention was seen as too complex or burdensome. These concerns may have reduced investment in co-creation leading to the limited engagement in the process.

Lack of Unified Implementation Plan

The implementation plan for a project of this magnitude and regarding a topic of such sensitivity posed a challenge within the consortium. Consortium members discussed a lack of clarity on the step-by-step process of co-creation in addition to complexities that were not fully accounted for and created barriers to implementation. The consortium accounted for a variety of languages within its national contexts (e.g., Spanish and Catalan in Spain, Italian in Italy, English and Ukrainian in Ireland, Greek in Greece, Swedish and Arabic in Sweden, and Bulgarian in Bulgaria). However, professional interpreters to support discussions and interviews with children was not consistently included. This language barrier further extended to implementation site staff who found identifying needs of the children in their sites to be challenging as they did not share the same language:

… [implementation site] staff says, you know, ‘insofar as we can communicate with the students, they seem to be OK.’ But that that lack, you know, the communication barriers around language made it difficult for them to probe or to get any deeper information. A teacher said, you know, ‘with other children you’d have more kind of conversations, so you’d be able to get more information or or kind of figure out how they’re doing in terms of their well-being. But because of this difficulty with oral language, it can be tricky to to ascertain that with migrant and refugee students. So you’re really just depending on their demeanor.’

Additionally, the implementation plan may not have fully accounted for the complexity of the needs, the power dynamics between stakeholders, the lack of clear steps for completing co-creation including explicitly stating what part of the chosen interventions could and could not be adapted, and the potential harm of discussing trauma for both stakeholders and researchers. When responding to the survey prompt regarding improvements to the needs analysis process, a consortium member shared:

We need to taking into consideration that interviews need to resemble as less as possible to asylum interview questions. In our opinion the procedure can be harmful to both parties if trauma is been revived.

A unified implementation plan for co-creation would need to plan for the multiple complexities that stem from refugee experiences.

Discussion

The current process evaluation provides insights into barriers and facilitators of the early phases of the REFUGE-ED project and, more broadly, into co-creation with refugee youth. While the CFIR framework was utilised to provide structure and support generalisability of findings, project constructs do not act in isolation to each other as demonstrated by the overlap seen within the analysis. Tracing the connections between the constructs and themes identified within the project and integrating previous findings can create a better understanding of the conditions for co-creating MHPSS and educational interventions with refugee youth.

The role of relationships is seen throughout the analysis as having potential to act as both a facilitator and barrier to co-creation. The current data showed a parallel between relationship building in the co-creation context to relationship building in the therapeutic context outlined by Bordin. Bordin (1979) proposed that therapeutic relationships were based upon shared goals, a division of tasks, and a bond between the therapist and patient built on trust. These core components to relationships can be seen across the project facilitators at all CFIR levels. Themes that oppose these components of relationship building are also seen in the current analysis as barriers across CFIR constructs. Namely, a lack of unity, differing priorities, and, figuratively, not meeting participants where they are at can be seen as working against or preventing the formation of relationships. The relationship between the researcher and participant was seen as a key tenant of the project design (Sordé-Martí et al., 2023) and the current analysis indicates the potential benefits. Specifically, ‘allies’ located within implementation sites provided externally located researchers with connection to the site and critical context to support the ongoing collaboration. Therefore, the importance of relationships can be seen in bridging gaps between researchers and implementing partners.

In addition to emphasising the importance of relationships, the current analysis indicates specific consideration that needs to be given to relationships in the implementation of co-creation. The current findings align with previous co-creation research that emphasised the importance of trust for the process of co-creation to be possible (Filler et al., 2021; Vettraino et al., 2017). Previous participatory community-based research found that the trust building process requires time and reflection to reduce power imbalance within the collaborative relationship (Wallerstein et al., 2019). Other co-creation research has suggested the value of investing in relationship-building early and throughout the process of co-creation (Afifi et al., 2011; Frounfelker et al., 2020). This investment in trust may counter some of the difficulties experienced within the REFUGE-ED project such as ‘not meeting participants where they are at.’

Further challenges, such as differing priorities, should be anticipated within co-creation research and strategies should be identified to address these competing demands. Within co-creation, choices are made regarding the balance of how much to direct participants in their identification of needs and creation of solutions. For example, Brown and Choi (2018) used an art methodology to allow refugee youth to co-create wellbeing supports with as few restrictions as possible. Brown and Choi did not plan to implement these solutions but rather used the co-creative process as a form of intervention. In contrast, the current study used co-creation to design interventions to be implemented. During this process, stakeholders and refugees often expressed divergent and broad needs creating challenges in how much could be addressed within the project’s scope. A clearer and more concrete understanding of REFUGE-ED’s scope for intervention may have narrowed the focus groups’ discussions and helped manage participants’ expectations. This may have also supported the development of shared goals and tasks, an established correlate of a therapeutic bond (Bordin, 1979). Future co-creation efforts should consider this balance in the design and implementation of the co-creative process with specific consideration for how this will be communicated to participants.

Relationship building with refugee communities and refugee youth further requires unique considerations. Potential traumas experienced on the migratory journey may impact mental health to the degree that functional impairments in relationships may be observed (Bronstein & Montgomery, 2011). Furthermore, migration experiences may create challenges in trust building, particularly with adults (Lustig et al., 2004). Trust building may be particularly challenging within a research relationship due to potential skepticism towards research caused by previous betrayals felt within research processes (Abdi et al., 2022). These challenges increase the importance of concerted efforts to build relationships and trust in co-creating with this group. Not only does this benefit the research, but previous evidence of the benefit of positive relationships formed between refugee youth and the community during research (Frounfelker et al., 2020) indicates the potential for positive effects of co-creation on refugee youth participants. This suggests potential benefits of relationship building not just for the research process, but for the wellbeing of refugee youth overall.

An additional concept that can be traced through each CFIR construct is the challenges in project planning and their effects on implementation. Specifically, barriers in Teaming such as a lack of conceptual unity and planning may have had consequent effects on other CFIR constructs culminating in the lack of a unified implementation plan. These challenges mirror the “operational challenges” described by Bustos and colleagues (2024). To overcome these challenges, Bustos and colleagues emphasise the importance of power balance and shared decision making. Within the current study, it appears that these operational challenges were underscored by a divergent understanding of co-creation between REFUGE-ED consortium members. The size of the current project may have contributed to this lack of conceptual unity as consortium members held different understandings of co-creation both conceptually and methodologically which led to discussions in consortium meetings in an attempt to reach agreement across the wide range of researchers. However, the consortium size also had potential benefits due to the breadth of expertise brought to the project. The balance between these concepts may present challenges for future research, particularly for projects that aim to scale-up co-creation. Thus, building on the proposal of Bustos and colleagues regarding the role of power in operational challenges, future co-creation projects that aim to scale-up the methodology will need to devise strategies to achieve a shared project understanding within the team which may then support shared decision making and reduce the influence of operational challenges.

The challenges to conceptual unity seen throughout the current analysis may reflect a broader issue within co-creation hindering its use. In their guidelines, the British Medical Research Council (BMRC) called for a need to understand the causal assumptions and mechanisms of an intervention (Moore et al., 2015). This understanding needs to explicitly account for adaptation creating clear guidelines regarding what elements of methodology can, need to be, and should never be adapted (Harker Roa et al., 2023). While other methodologies may have established some of these principles, co-creation may be lacking a unified operationalisation across the field (Halvorsrud et al., 2021) which may be reflected in the current study. Afifi and colleagues (2011) demonstrated the potential for co-creation to be used with refugees to develop a logic model for an intervention. Through co-creative methods, the participants designed an intervention while outlining the assumptions, inputs, activities, outputs outcomes, and impact of the intervention. A similar process in this project may have clarified some of the key project assumptions and mechanisms, leading to greater unity in implementation.

Planning and implementing co-creation with refugee communities also needs to account for the needs specific to this community. The complexities of the refugee crisis and the effects of the crisis on the community (Weine, 2011) requires concrete planning. Some of these complexities stem from cultural context and require adjustments of interventions to align with participants’ culture while understanding this is not a monolith (Nakkash et al., 2012). Other challenges may be related to the systems that refugee youth exist within. For example, the system rigidity, lack of buy-in to the process, and challenge in accessing refugee youth acted as barriers in this research. While all barriers to co-creation cannot be predicted, these challenges should be anticipated and planned for in advance of co-creation research to mitigate their harm. Given the potential benefits of the co-creation process for refugees (Filler et al., 2021; Simonsen & Ryom, 2023), it is vital that system barriers are addressed in implementation plans and mitigation strategies are designed in advance of co-creation research. Despite planning efforts, though, barriers may still arise. Flexibility and an openness to less traditional research methods have been found to support researchers through unexpected challenges that will arise from co-creative research (Home et al., 2021). Increased use of process evaluations in similar projects can document these planning and implementation efforts supporting the future use of co-creation.

The current findings should be considered in light of limitations. Firstly, the research team that conducted the process evaluation had a role in the implementation and outcome evaluation of REFUGE-ED. BMRC outline some benefits of these overlapping roles, including avoiding redundancy in evaluations, but also caution the potential for bias (Moore et al., 2015). While the role of the authors in the project may limit objectivity, it may also support the analysis as the authors have a thorough understanding of the project context. Further, the involvement of multiple authors in the analysis process and the use of an a priori analytical frame grounded in implementation research may counter potential bias.

A second limitation of the current analysis is the lack of inclusion of the target population, namely refugee youth. Given that a core principle of co-creation is the involvement of target populations in the research process, the inclusion of refugee youth in evaluating the process of the current project would have supported the triangulation of data and may have strengthened the current findings. However, the aim of the current study was to evaluate co-creation from the perspective of intervention implementors. As such, consortium members and implementation site staff were the primary focus. The inclusion of consortium members and implementation site staff may have helped to identify barriers and facilitators at an organisational level providing findings relevant to planning co-creation within other projects. Future research should expand upon these findings by including refugee youth to understand their experiences of co-creation, however, consideration for the potential burden on participants and barriers to engagement and prioritisation of research raised within this study would need to be addressed.

While there were limitations to the current study, the methodology and analysis strengthen the relevance of the findings. Firstly, the current paper includes reflections on the co-creation process that were collected both during and after the needs assessment phase. The use of data collected throughout the project means that barriers identified did not necessarily remain unresolved. For example, a challenge expressed throughout team meetings was the difficulty in engaging children in the co-creation process. However, a review of the project’s implementation shows that in most cases this was able to be overcome and children were eventually engaged in the co-creative process. However, this information was not known at the time of data collection. Thus, the barriers and facilitators identified should not be interpreted as tasks that were or were not completed during the project but rather as concepts that supported or worked against co-creation. The analysis of data collected throughout the co-creation process provides future research with information on what to anticipate. This data is further complimented by data collected at the end of the co-creation phase which allowed for retrospective reflection on the process as a whole. The inclusion of both types of data can better inform future co-creation projects.

Further, the use of broad questions during the data collection phase strengthens the analysis conducted. The data collection utilised general questions (e.g., ‘what facilitated the co-creation process?’) to ensure that the study participants identified the most salient facilitators and barriers. The use of deductive analysis through the incorporation of CFIR provided a way to organise the results while the use of inductive analysis to generate themes allowed for themes to be created based on participants’ experiences. This combination of methodologies ensures that results are both highly relevant to researchers and organised in a way that supports their comparability to other findings.

Overall, the analysis conducted adds to the current understanding of process evaluations, co-creation, and co-creation specifically with refugee youth. The use of CFIR demonstrates the interconnections between project constructs wherein barriers and facilitators in one element of the project have interacting effects to other elements. Recognising these connections suggests that focusing on specific project elements, namely building relationships and developing implementation plans based on unified concepts, can support co-creation with refugee youths in a variety of ways.

Conclusions

Understanding co-creation and the needs of refugee youth requires specific decisions within research collaborations. Research collaborations need to make explicit and justified decisions regarding the mechanisms for implementing co-creation that consider the research context. In addition to the researcher, though, the systems that research occurs within need to account for the needs of co-creation when it is chosen as a methodology. This includes grant cycles, institutions, and research partners understanding that co-creating with refugee youth requires specific steps that may increase the time and resources needed for relationship building and planning. As this methodology and these systems develop, process evaluations can continue to shed light on methodological considerations for successful co-creation.