Background

Clinical decision-makers and policy and guideline developers often use the results of systematic reviews of randomised trials and other studies, to guide and inform healthcare practices. Randomised trials have long been considered the gold standard for testing the effectiveness of interventions, yet they are often wrought with challenges. One challenge is that of slow or suboptimal recruitment, with reports suggesting that about half of all trials do not meet their recruitment target or do so only with an extension to the original trial duration [1, 2]. For example, of 73 trials funded by the United Kingdom (UK) Medical Research Council (MRC) and the Health Technology Assessment (HTA) programme between 2002 and 2008, 55% of trials recruited to their target sample size and nearly half (45%) received an extension [3]. Similar issues have been reported in the United States. A study investigating the prevalence and associated economic impact of low enrolling clinical studies at a single academic medical centre found that of the 837 clinical studies terminated during the study period, 31.1% were closed because of low recruitment and at a cost of almost $1 million [4].

Under-recruiting or stopping a trial early due to poor recruitment has major implications for the study outcomes, not least a reduction in the study's statistical power [5]. Underpowering a trial adds uncertainty; for example, an underpowered study may report no difference between groups on clinically important outcomes when, in fact, a difference may exist. Other implications of poor recruitment or stopping a trial early include increased burden and resource waste, ethical issues, and reduced impact on clinical care as the research question may be insufficiently addressed to inform clinical practice [1, 6].

To boost recruitment or address slow recruitment, trial coordinators often engage in responsive activities, for example, altered or increased communication strategies [7], incentives [6], or formal site visits by the principal investigator [5], yet sufficient, robust evidence on the effectiveness of many of these activities is lacking. Uncertainties around trial elements that might potentially impact recruitment have also been explored in survey research [8,9,10]. To further explore and, importantly, prioritise trial recruitment uncertainties, the Health Research Board Trials Methodology Research Network (HRB-TMRN), in 2016, undertook the PRioRiTy study [11]. Using a James Lind Alliance-Priority Setting Partnership (JLA-PSP) approach, the PRioRiTy study identified and ranked the Top 10 priority questions for trial recruitment uncertainties. One thematic area that emerged in PRioRiTy was educating and training trial recruiters. This finding also reflects a ranking exercise involving members of UK Clinical Trial Units (CTUs), where training site staff was identified as the number one priority for future evaluative research [12].

Training recruiters has been found to improve enthusiasm for trials and build recruiter confidence in communicating about trials with patients [13]. Yet, evidence of the effectiveness of trial recruiter education and training interventions, and the types of training required, appears lacking [14]. For this reason, the Training tRial recruiters; An educational INtervention (TRAIN) project was established to develop and assess the acceptability of an education and training intervention for recruiters to neonatal trials. Acknowledging that all trials can experience recruitment challenges, we specifically chose neonatal trials as the focus for TRAIN because recruitment challenges to these trials can be compounded further by having to approach parents at a challenging time, that is, in the context of parental fear, worry and concern for a new baby who may be very unwell, and within a time scale that is often limited when making a decision [15].

Methods

Aim

We aimed to develop and assess the acceptability of an education and training intervention for recruiters to neonatal trials. The study involved three sequential phases to achieve this aim, with each phase contributing information directly to the subsequent phase(s). These phases were 1) evidence synthesis, 2) intervention development and 3) intervention acceptability assessment (Fig. 1).

Fig. 1
figure 1

TRAIN project phases

Full size image

Phase 1: Evidence synthesis

The purpose of phase 1 was to gain preliminary information that could be used to inform aspects of phase 2. The evidence synthesis activity involved a systematic review of the effectiveness of existing training interventions, a content analysis of the content, format and delivery of existing training interventions, and a concept analysis of the concept ‘trial recruitment’. The findings of phase 1 have been reported elsewhere [14, 16, 17]; thus, in this paper, we report the conduct and findings of phases 2 and 3.

Phase 2: Online survey of recruiters to neonatal trials

To ascertain the opinions of recruiters to neonatal trials on the specific education and training requirements that they believed would enhance trial recruitment and to provide data from the perspectives of ‘recruiter’ stakeholders to further inform the development of TRAIN, an online survey of recruiters to neonatal trials was conducted. The survey was designed using the findings from phase 1, especially the findings of the content analysis which provided information on recruiters’ preferences around training delivery format, training materials, duration of the training, and training content [17]. The draft survey was subjected to validity assessments by a panel of five expert trial recruiters, after which minor refinements were made. The final survey (Additional file 1) included a mix of multiple-choice, Likert scale and open-ended questions with free text boxes for comments.

The target population for the survey was all individuals involved in recruitment to neonatal trials, either directly or indirectly (i.e., in designing recruitment processes) across Ireland and the UK. As the number in the target population was unknown, we were unable to determine a study sample size estimate; however, our intention was to gain as many responses as possible. The survey was distributed online using the QuestionPro platform in November–December 2020 for four weeks, with a reminder sent at the end of week two. A purposive sampling approach was used, supplemented by snowball sampling, whereby the survey was advertised by email and on social media via neonatal trial networks, clinical trial units, neonatal trial research facilities, and at a neonatal research symposium (Nov 2020, Ireland), with a request to forward the survey to known others who were involved in recruitment to neonatal trials. Survey distribution was also supported by the National Perinatal Epidemiology Unit (NPEU) Clinical Trials Unit, the Irish Centre for Maternal and Child Health Research (INFANT) and the HRB-TMRN.

Phase 2: Intervention development co-design/co-production workshops

The final stage of developing TRAIN involved Partnership methodology using co-design/co-production methods. The TRAIN Cooperative Intervention Development Committee (TCIDC) was established and met with members of the core research team in two arranged workshops to draft TRAIN. TCIDC members were purposively selected based on their expertise. Members were two neonatal clinicians, two neonatal research nurses, one neonatal trial manager and four Public and Patient Involvement (PPI) contributors (parents of neonates previously involved in a neonatal trial) from the UK and Ireland. The TRAIN research team initially proposed a draft of TRAIN based on the evidence syntheses and the online survey findings. This draft was shared with the TCIDC for review before the first workshop. The intervention development workshops took place in March and April 2021 (online via Zoom due to COVID restrictions). The workshops were of 2-h duration and, with informed consent, were recorded and transcribed (for memory and recall purposes only).

Phase 3: Acceptability of TRAIN

In phase 3, TRAIN was delivered to neonatal trial recruiters to assess for acceptability and to gain their feedback. The target sample included all individuals who had ever been, or would be in the future, involved in recruiting to a neonatal trial. An invite to participate in the acceptability workshops was issued during October–November 2021 to international neonatal trial recruiters by email and social media, via neonatal trials networks, clinical trial units and neonatal trial research facilities, and via the NPEU Clinical Trials Unit, the INFANT Centre, and the HRB-TMRN. The invite included a link to register for one of three training dates in November 2021 which involved attending a 2-h online TRAIN intervention workshop and completing a five-minute before-and-after survey. Once participants registered for a training date, they were sent a confirmation email with a link to the online training and the baseline survey.

The education and training sessions were delivered via Zoom, and each training session was facilitated by two members of the TRAIN core research team (AH, VS, HD). As the participants were involved in recruiting to different trials, template content that outlined what would be presented in Unit 1 for specific trials was presented, rather than actual content. Examples of training resources were also shared with participants, thus allowing them to provide feedback on the content of Unit 1. Units 2 and 3 were presented as they would be in a real training scenario. At the end of each Unit, participants had an opportunity to share comments or feedback on the Unit and were reminded that further comments could be shared in the online follow-up survey.

The pre-and post-session surveys were designed to capture three outcome measures. These were recruiters’ perceived preparedness and self-confidence as neonatal trial recruiters, recruiters perceived rating of their knowledge of the trial information, and recruiters perceived satisfaction with the training intervention. The pre-and-post surveys (Additional file 2) included a mix of multiple-choice, Likert scale, and open-ended questions with free text boxes for participants to include any other information.

Ethical conduct

Ethical approval for the study was granted by the School of Nursing and Midwifery, Trinity College Dublin, Research Ethics Committee, Ref: 14th May 2019 (see ethical approval and consent to participate section for full details). The study was ethically conducted in accordance with the General Data Protection Regulations (GDPR) (2018) (https://gdpr-info.eu/).

Results

Online survey of recruiters to neonatal trials

Ninety-three recruiters responded to the survey representing clinicians involved in front line recruitment (n = 34, 37%), principal investigators (n = 24, 26%), trial managers (n = 18, 19%), researchers involved in frontline recruitment (n = 10, 11%), trial methodologists (n = 4, 4%) and other (n = 3, 3%). Respondent’s experience in neonatal trial recruitment was < two years’ (n = 13), 2–6 years (n = 23), 7–10 years (n = 24) and > 10 years’ experience (n = 33).

Not all responders answered every question. Of 78 respondents who responded to item 12 which sought information on ‘the location of the most recent neonatal trial you were involved in recruiting to’, 74% indicated the UK, 26% the Republic of Ireland and 9% elsewhere. A high proportion of respondents (87%) agreed that it would be helpful to receive training and education about neonatal trial recruitment, even though most (83%) had previously received such training. Of those that had previously received training, 32% indicated that they had received training specific to neonatal trials, and 64% had received training about trials in general.

Respondents were asked to rank a list of eight training delivery methods in order of preference. The preferred method was a face-to-face presentation or lecture format, followed by webinars, one-to-one support in practice, and through practice such as roleplay (Fig. 2). Post-training refresher sessions were ranked low by participants, however, one respondent commented that these sessions should be provided regardless of the method of training originally provided. Respondents held mixed views as to the optimal duration of training sessions (Fig. 3).

Fig. 2
figure 2

Preferred method of education and training delivery

Full size image
Fig. 3
figure 3

Duration of training sessions

Full size image

Regarding supportive education and training materials, practical checklists and top tips documents were the most popular, followed by lecture notes/slides and template recruitment materials. Reading lists or reading material was the least preferred option.

A list of sixteen trial recruitment topics was provided to respondents who were asked to rate these on a five-point scale from 1 = extremely beneficial to 5 = not at all beneficial. The top three topics in the ‘extremely beneficial’ category included background information on the study (65%), informed consent (65%), and participant eligibility (56%) (Table 1).

Table 1 Aspects of trials that would be beneficial to have training on (data from phase-2 survey)
Full size table

Respondents also had an option of adding any other aspects of trial recruitment that they thought would be beneficial for training. Fifteen participants responded to this item with topic areas including communication skills (building rapport with parents, approaching distressed parents, building empathy), PPI in trial design and training design/delivery, trial monitoring, and embedding trials as a research culture in a unit. These responses also reflect free-text comments provided by respondents in relation to the barriers and facilitators to trial recruitment, which were coded and organised into seven representative categories (Table 2).

Table 2 Perspectives on barriers and facilitators to recruitment (based on data from phase-2 survey)
Full size table

TCIDC workshops

During Workshop 1, the TCIDC shared their feedback and experiences and new ideas and recommendations for TRAIN. The TCIDC participants emphasised the importance of building a rapport and communicating empathetically with parents when inviting their infant to take part in a trial, and that this element should feature as a significant component of the TRAIN intervention. During workshop 1 there was considerable discussion around developing tools to help recruiters see the recruitment scenario from a parent’s perspective, which was deemed challenging given their busy caseloads. Summary recommendations arising from workshop 1 are provided in Table 3, with additional detail provided in Additional file 3.

Table 3 Summary recommendations from Workshop 1 (based on workshop participant’s input)
Full size table

The research team analysed the discussion transcripts from Workshop 1 and collated these in an overview table. This helped map the proposed changes and new ideas to the existing draft intervention and further helped refine and develop the intervention. The updated draft intervention was circulated to the TCIDC before Workshop 2. During the second workshop, the TCIDC provided their final feedback and recommendations. Recommendations from Workshop 2 included points mainly related to the intervention resources' order, structure, and format. Following workshop 2 TRAIN was finalised and prepared by the research team for acceptability testing.

TRAIN

The resulting intervention has five primary learning outcomes, such that, by the end of the intervention recruiters attending the education and training session will:

  1. 1.

    Understand the trial protocol and be able to explain to parents what taking part will involve

  2. 2.

    Understand and be able to explain the process of randomisation to parents

  3. 3.

    Be aware of factors to consider when approaching parents for recruitment of their neonate to a trial

  4. 4.

    Understand and be cognisant of parents’ perspectives when recruiting their neonates to a trial

  5. 5.

    Be prepared to engage in a recruitment conversation

TRAIN is designed with the intention that the training is offered to recruiters before trial recruitment begins, although the Units (Table 4) can be provided throughout trial recruitment as refresher sessions as necessary. TRAIN can be delivered online or in-person/face-to-face. A detailed trainer manual describing specifically how TRAIN should be delivered was compiled alongside the intervention resources. Once evaluated for effectiveness in a future definitive intervention trial, and finalised, the intention is that a representative from any trial team can follow TRAIN’s manual guidance in providing the education and training independently. TRAIN’s three core learning units are:

  • Unit 1: The trial protocol (50 min)

  • Unit 2: Understanding randomisation (5 min)

  • Unit 3: Approaching and engaging with parents (70 min)

Table 4 Overview of the TRAIN intervention and resources
Full size table

Unit 1 is trial-specific and focuses on the trial protocol. Units 2 and 3 are generic with applicability to any neonatal trial. Table 4 presents an overview of each unit and the related resources.

TRAIN acceptability assessment

A total of 11 recruiters to neonatal trials registered to attend an acceptability training session. These 11 were distinct from those who participated in the phase 2 survey as the invite for the training sessions was extended globally. Of the 11 who initially registered, seven did not attend. Thus, only four recruiters took part, two in each of the two training workshops. These recruiters were from Italy (n = 2), Ireland (n = 1), and the United States (n = 1).

Feedback on TRAIN was positive. All four participants commented on the value of the training Units and resources, especially the ‘recruiter lanyard’ resource in Unit 1 and the parent video vignettes in Unit 3. Participants commented that if there was more flexibility in how the training is facilitated (e.g., training divided across workshops of shorter duration, rather than in one workshop), this would likely make it more accessible and enabling for recruiters to attend. The current set-up of two hours was also challenging given the depth of material being covered, and the attention required to assimilate the information; thus, it was suggested that the Units should be delivered in two sessions; Unit 1 on its own and then Units 2 and 3 together.

The recruiters suggested that Units 2 and 3 would be beneficial to incorporate into Good Clinical Practice training or as part of induction training for new staff joining neonatal units. Participants also commented that the role play session in Unit 3 was only appropriate if all TRAIN participants were recruiting to the same trial. One also noted that facilitating this session would require a particular set of facilitation skills that could not be learned from guidance in the training manual alone. One other suggestion from the participants included translating the training and resources to multiple languages to improve its accessibility.

The baseline survey, completed by all 11 who registered for a workshop, indicated that the clinical setting or trial site was active or extremely active in recruiting neonates and their parents to participate in neonatal trials for 72%. When asked about the level of support provided for the recruiters in their clinical setting/trial site, 64% reported setting/trial sites as supportive or extremely supportive, 27% were not supportive/not at all supportive, and 9% were unsure. When asked about recruiting participants to a neonatal trial, 50% of respondents felt prepared or extremely prepared, 42% were unsure, and 8% felt not at all prepared. Regarding recruiter confidence and knowledge, 75% felt confident or extremely confident, 17% were unsure, 8% were not confident, 67% felt knowledgeable or extremely knowledgeable, and 25% were unsure.

All four recruiters who took part in TRAIN completed the follow-up survey, and all four responded that following the training they felt prepared, confident or extremely confident, and knowledgeable or extremely knowledgeable. When asked to rate each of the elements of the TRAIN intervention on how useful to neonatal trial recruitment they perceived them to be, one was unsure about the duration of the training, and all four rated the remaining elements as extremely useful or valuable.

Discussion

Poor recruitment has negative consequences and trial extensions are costly. When recruitment targets are not met, research questions are left unanswered, wasting money and participants’ time. TRAIN, underpinned by evidence syntheses and an engaged Partnership approach to intervention development, offers a comprehensive training and education intervention for recruiters to neonatal trials. The package is unique in that the views of recruiters on their education and training needs as well as the voices of those being recruited critically informed the intervention. Although recruiter training activities exist in various forms [17] evidence for the effectiveness of recruiter education and training is limited [14] and no studies were identified that focused specifically on training recruiters to neonatal trials. Furthermore, while several other recruiter training interventions, albeit in other healthcare areas, were designed for recruitment to a specific host trial [18,19,20], TRAIN offers both trial specific (Unit 1) and trial generic training (Units 2 and 3). This will be important for reducing trial team burden with regards to recruiter training resource development in future neonatal trials. Furthermore, TRAIN was developed such that the Units can be offered prior to trial commencement, implemented as a refresher, or implemented in response to recruitment issues during the trial, thus offering considerable flexibility to trial teams in how they might use TRAIN. This also distinguishes TRAIN from other training interventions, with several implemented in response to recruitment issues in an ongoing trial only [21,22,23,24] or embedded from the start of the trial [25,26,27,28].

A considerable portion of TRAIN (Unit 3) focuses exclusively on approaching and engaging with parents of neonates about the trial. Communicating information about a trial is important for any trial participant. This, however, has further significance in the context of neonatal trials. For example, Duley and colleagues [15] report that parents often declined to take part in perinatal trials due to burden, inconvenience, or worries about risks to their baby. Parents wish to feel informed yet report that the time provided to decide was insufficient, especially as many also wanted input from others before making their decision. Critically, Duley and colleagues described how parents reported that being approached to take part in a trial compounded their stress and worries, especially if they were approached at an inappropriate time [15]. Parent contributors in the TCIDC reinforced these findings and emphasised the importance of recruiters building a rapport with parents prior to initiating trial recruitment discussions, as well as stopping to pause and consider whether the timing of the approach was appropriate.

When recruiting to neonatal trials, recruiters also need to be cognisant of the maternal parent having recently given birth. For example, a qualitative study that explored the experiences and views of women (n = 22) and healthcare professionals (n = 27) of recruitment to a trial [29], found that while clinical staff placed emphasis on imparting information in clear and succinct ways, the reality of their post-birth situation for many women led them to make quick decisions without full engagement or an understanding of the potential risks of trial participation. For this reason, women suggested that information about a neonatal trial should be provided in the antenatal period to help ensure that consent to take part is informed [29]. Arguably, this approach could produce opposing effects, such as causing undue concern or worry, especially as many women will not be approached if the trial is focused exclusively on neonates who are unwell.

Strengths and limitations

The comprehensive development of TRAIN is a particular strength in this study, with all involved stakeholders contributing to the final version of TRAIN. Although a considerable depth of data was provided by 93 respondents to the survey, we are unclear if this number reflects a sufficient sample size to affect representation as the precise target population number is unknown. Importantly, however, the respondents to the survey represented all categories of recruiters (i.e., front-line clinicians, trials managers, etc,.) and with a range of years of experience in neonatal trial recruitment. Securing recruiters for TRAIN’s acceptability assessments, however, was challenging, with four of 11 recruiters only taking part in the assessments. Nonetheless, the recruiters that did take part provided valuable insight into the acceptability of TRAIN, and provided meaningful recommendations for refinement, for example, having greater flexibility in the format for delivering TRAIN and facilitating training Units across two rather than one session. It would have been beneficial, however, to directly match individual pre- and post-training survey results, however, for reasons of confidentiality and anonymity, only aggregated results are available. The accrual of seven recruiters to the acceptability assessments highlighted difficulties that can exist with recruiter training engagement, especially when many recruiters will also hold busy clinical roles. Engagement of recruiters in trial training workshops has been identified in previous research and suggestions to mitigate low engagement discussed (e.g., personal engagement, encouragement by the study chief investigator to attend, and a long lead-in time for training) discussed [29]. These issues will require consideration in any future TRAIN scale-up, piloting, and effectiveness testing.

Conclusion

TRAIN offers a comprehensive co-produced training and education intervention for recruiters to neonatal trials. TRAIN was deemed acceptable, with minor modification, by neonatal trial recruiters. Scale-up of TRAIN with formal pilot testing and testing for effectiveness in a large cluster randomised trial is required. Future intentions for this would be to embed TRAIN as a Study Within A Trial (SWAT) in a future pan-European or International cluster trial whereby half of the recruiting sites involved would receive TRAIN and the others standard training or trial training as usual for the site. To support scale-up, however, funding will be required; thus, grant calls that specifically support comparative intervention or SWAT research will be targeted.