Introduction

Equitable representation by sex and gender in clinical trials is an essential step to ensuring true patient-centered, evidence-based care. An individual’s sex, a determination made at birth based on an individual’s biology, and gender, a construct based on an individual’s lived social and cultural experiences, each has the potential to influence the effect of an intervention or natural history of an illness. In the current era of precision medicine and personalized medical care, failure to incorporate an evidence-based understanding of the influence of sex and gender on an individual’s health represents a missed opportunity to optimize outcomes and risks an adverse health event. For example, women experience much higher rates of adverse medication side effects related to sex-based differences in pharmacokinetics, yet for most commonly used medications, the original clinical trials supporting their approval did not report sex-stratified outcomes [1]. In order to be able to generate sex and gender-specific science, we need sufficient numbers of women in trials to support the statistically sound exploration of differential treatment effects.

Recent decades have seen the implementation of multi-national legislation to overcome the historical discriminatory exclusion of women from trials [2]. In the 30 years, since the NIH established the Office of Research on Women’s Health (ORWH), multiple programs have spurred new investigations into women’s health, funding to support the career development of women investigators, and the development of a rich array of tools to support the inclusion of sex and gender in clinical research [3]. In 1998, the FDA established a mandate to include both men and women as well as sex-based analyses for trials supporting the approval of drugs intended for both sexes [4]. While actions like these have led to improvements and near parity in sex-based participation in many drug trials [5], inequity persists in important areas. In particular, women continue to be underrepresented in trials related to cancer [6], chronic kidney disease, vascular disease [7], and certain cardiovascular conditions [8,9,10,11,12].

To increase the proportion of women participating in clinical research, evidence-based strategies are needed to enhance the inclusion of women in trials. Prior work has explored how certain study design features (e.g., blinded intervention assignment) influence trial participation rates generally [13], but we know less about what other trial design features (e.g., population engagement in trial design) and study conduct approaches (e.g., gender concordant study staffing) have been deployed to increase participation by women specifically. In particular, understanding how study teams have strategically approached trial activities with the purpose of promoting adequate representation of women could inform future trials as they attempt to address participation disparities. Specifically, we sought to develop a broad understanding of where there is evidence about approaches deployed to include women across the lifespan of a trial. Thus, we conducted an evidence map to answer the following question: What strategies have been described or evaluated to boost the enrollment or retention of women in clinical trials?

Methods

We selected evidence mapping methodology as it is appropriate for reviews that seek to describe the breadth of a body of literature and identify areas for future research rather than focus on the specific effects of a narrow, defined intervention [14, 15]. We were unable to find a prior review assessing approaches to include women in trials, thus starting with developing a broad understanding was appropriate. We developed an a priori protocol that was posted online: (https://osf.io/cbhxt?view_only=cd368b067f2644869b504a83d156fae6) and followed reporting guidance from the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines where applicable (e.g., we did not evaluate risk of bias or estimate overall effect).

To support clarity, we established definitions for key terms (see Additional file 1). Of note, while we planned to report both sex and gender as presented within included studies, we found that the current literature did not report these constructs separately nor did they routinely define their use of terms such as “women”, “female”, “sex”, or “gender”. We acknowledge that this is conceptually problematic as it conflates the distinct dimensions of sex and gender. Due to this limitation of the existing primary literature, we use the term “women” to reflect any individuals reported by an article to be women or female from this point on. We note that this challenge has implications for research on both sex-based and gender-based differences [16].

Search strategy

We searched MEDLINE® (via PubMed) and EMBASE (via Elsevier) databases from inception to 4/4/2023. An experienced medical librarian (SC) devised and conducted the searches, with input on keywords from the other authors. We used a combination of database-specific subject headings and keywords related to women, recruitment, retention, and clinical trials. Editorials, letters, case reports, and comments were excluded. To increase specificity, pediatric-only literature was also excluded. The searches were independently peer-reviewed by an additional librarian using a modified PRESS Checklist [17]. The full, reproducible search strategies for all included databases are located in Additional file 2. In addition, we reviewed the references of previous systematic reviews conducted on related topics for potentially relevant references [18,19,20,21].

Study selection

We used prespecified eligibility criteria for both quantitative or descriptive (Additional file 3) and qualitative (Additional file 4) articles. Articles describing recruitment or retention strategies employed during the conduct of a clinical trial with the intent to increase the inclusion of women and which targeted participants, study staff, or investigators were eligible. We excluded articles describing trials conducted outside of countries recognized by the Organization for Economic Co-operation and Development (OECD) [22] and those not available in English. Article titles and abstracts were reviewed by all co-authors (except SC) for potential relevance to the research question with one vote leading to inclusion for full-text review and two for exclusion. At the full-text screening, pairs of investigators agreed on the final article disposition status. Disagreements were resolved by consensus or by obtaining a third reviewer’s opinion (KMG). Covidence (Veritas Health Innovation, Melbourne, Australia), a web-based software that streamlines literature and systematic reviews, was used for screening and tracking screened and included articles [23].

Given the small number of relevant articles describing recruitment and retention outcomes, we also included those with narrative descriptions of strategies in a trial that did not include specific outcomes of interest or used a study design not appropriate to evaluate strategies (i.e., non-comparative designs). Given the large number of articles identified describing trials, we prioritized these over self-identified pilot studies or feasibility trials. We considered the unit of analysis to be the article rather than the trial as some articles described strategies used across more than one trial. Additional file 5 lists articles excluded at the full-text review stage and the reasons for exclusion.

Data abstraction

Data abstraction was conducted by two sub-teams: one focused on study characteristics and the second on described strategies impacting recruitment and/or retention. The first sub-team (SAD, JS, CB, RL, DP, CS) abstracted high-level study characteristics using a form developed in Covidence for data abstraction. Study characteristic abstraction was piloted with each reviewer abstracting data from 6 articles in round 1 of piloting and 10 articles in round 2. Abstracted characteristics included information to provide context for study strategies such as patient descriptors (e.g., age, sex/gender, race/ethnicity), intervention characteristics being studied in the trial (e.g., disease condition, invasiveness of intervention), trial design (e.g., comparator, study setting), stated recruitment goals, and final recruitment numbers.

The second sub-team (AKD, LCK, KMG, KS, MSB, DP, CS) abstracted study-specific recruitment and retention strategies into a REDCap form [24]. To pilot this form, each reviewer was assigned the same 4 articles to pilot for the first round and 20 articles for the second round. After each round of piloting with both groups, results were compared and discussed, and the abstraction forms were modified. Subsequently, data from each included article was abstracted by at least one reviewer. Twenty percent of each reviewer’s abstractions were over-read to verify accuracy. For those reviewers whose abstraction quality was considered insufficient based on the frequency of errors of either commission or omission, the entirety of that reviewer’s data abstractions were over-read and corrected as needed. We approached strategy abstraction by trial phase: trial development (e.g., patient and community partner engagement, trial staff training), participant outreach (e.g., location, modality, and partners for potential participant outreach), enrollment (e.g., flexible modality and location for consent), intervention delivery (e.g., flexible timing for intervention delivery) and outcomes assessment (e.g., remote data collection), analysis (e.g., recruitment/retention by gender/sex), and dissemination (e.g., plans to share trial results with participants or community).

Assessment of methodological quality of individual articles

As this is an evidence mapping review, we did not assess the methodological quality of individual articles or conduct certainty of evidence ratings.

Data synthesis

We narratively summarized the study characteristics of the identified literature using relevant data abstracted from the eligible articles. We organized trial strategies by trial phase as described above and then by level of targeted action (e.g., study participant, source community, study team). We then looked for patterns across articles related to types of recruitment/retention strategies employed in relationship to characteristics such as type of intervention (e.g., pharmacologic vs. behavioral), whether the condition studied was sex-specific or not (e.g., pregnancy vs. cardiovascular disease), and population studied. In particular, we considered reporting patterns of specific strategies by articles focused on the recruitment of women from historically marginalized racial/ethnic populations. Descriptive statistics about the included studies were calculated in Microsoft Excel [25].

Results

We identified 122 eligible articles (see Fig. 1) reporting on recruitment and retention strategies for 136 trials (total n = 377,595 women; median 285 per trial). Of the 122 articles, 95% recruited only women. Seven articles reported on trials that recruited participants other than women: 1 recruited couples, 3 recruited men and women (not couples), and 3 recruited women and children. The majority of articles reflected trials conducted in the USA (64%). Fifty-eight articles provided narrative descriptions of strategies employed, 72 provided some degree of comparison between strategies used either within a study or between studies, and 10 articles reported qualitative data collection from either trial participants or study staff about trial recruitment experiences (see Additional file 6 for a detailed description of included articles). Ninety-two (75%) of all articles addressed sex-specific conditions (e.g., peripartum conditions, gynecologic conditions) vs. 30 (25%) that addressed sex-neutral conditions (e.g., infectious diseases, cardiovascular disease). Few articles were relevant to conditions recognized to have an evidence base that underrepresents women; for example, we found only 6 articles describing strategies in cardiovascular trials and none relevant to sex-neutral cancers or chronic kidney disease. Behavioral interventions were most common (52%), followed by pharmacologic (23%), and only 4% were invasive in nature. The majority of the pharmacologic and invasive interventions were for sex-specific conditions (20 of 28 and 4 of 5, respectively). Forty-one articles described strategies for purposefully recruiting women from a specific racial or ethnic group with Black or African-American being the most common (n = 19) and 14 articles reported on strategies to include multiple racial/ethnic minoritized populations (Table 1).

Fig. 1
figure 1

Literature flow

Full size image
Table 1 Summary of included articles describing recruitment strategies for women
Full size table

Described strategies were most commonly reported during the stage of a trial in which the study team conducts outreach to potential participants (95%; Tables 2, 3, 4, 5, 6, 7 and 8). Outreach was the most predominant stage across all intervention types (Fig. 2). The most common strategy for this trial stage was the modality of recruitment advertisements which was reported in 98 articles (e.g., mass media, print letters, word of mouth), identification of community partners in the recruitment process in 74 articles (e.g., clinicians, laypersons, or peers), followed by the location for outreach efforts reported in 74 articles (e.g., clinics, community-based locations, churches), and tailoring of patient-facing recruitment materials by 40 articles (e.g., tailoring the language or images, or developing bilingual materials) (Fig. 3).

Table 2 Detailed strategies by trial stage: Trial Planning (56 articles)
Full size table
Table 3 Detailed strategies by trial stage: Participant outreach (116 articles)
Full size table
Table 4 Detailed strategies by trial stage: Enrollment (40 articles)
Full size table
Table 5 Detailed strategies by trial stage: Intervention delivery (76 articles)
Full size table
Table 6 Detailed strategies by trial stage: Outcomes assessment (21 articles)
Full size table
Table 7 Detailed strategies by trial stage: Analysis/interpretation (3 articles)
Full size table
Table 8 Detailed strategies by trial stage: Dissemination results (4 articles)
Full size table
Fig. 2
figure 2

Studies with at least one strategy in a study phase, by intervention type

Full size image
Fig. 3
figure 3

Strategy types across trial phases (n = number of articles)*

Full size image

The next most common trial phase was intervention delivery (62% of all included articles); specific strategies in this category included incentives and compensation (62 of 76 articles mentioning intervention delivery strategies), reduced intervention burden (36 articles), communication (28 articles), and study staff-participant relationship management (5 articles). Strategies relevant to trial planning were reported by 56 articles. Common groups of strategies were relevant to partnered engagement in trial design, structural planning (e.g., site selection considerations), and eligibility considerations. Forty articles reported strategies related to the enrollment process including flexibility around the consent process (e.g., modality, timing), adapted consent process, and open design. Strategies relevant to outcomes assessment were reported by 21 articles and were primarily related to reducing the assessment burden. Two articles mentioned the involvement of participant and community member engagement in results interpretation, 3 articles provided final trial results to participants, and 2 engaged with partners around dissemination.

When considering articles focused on intersecting identities, we found that articles about recruiting women from marginalized racial/ethnic populations more often reported strategies during the planning phase of trials (78%) and intervention delivery (75%) compared to articles without a specific population subfocus (30% and 55%, respectively). Finally, we considered strategies across the trial phase across both sex-specific and non-sex-specific conditions (Additional file 7). We found across all conditions that the majority of strategies fell in the first 3 trial phases (i.e., trial development, outreach, enrollment).

Discussion

The peer-reviewed literature evaluating and/or describing strategies to boost the inclusion of women in clinical trials is drawn primarily from trials of sex-specific conditions and most commonly described strategies pertaining to the process of outreach to potential participants. A smaller proportion of included articles discussed strategies used for recruiting women in trials evaluating invasive interventions or medications and almost exclusively focused on sex-specific conditions. There were notably fewer strategies described during study conceptualization and design, analysis and interpretation, or dissemination. Additionally, there was almost no literature describing approaches to increase the participation of women within trials that included both men and women, and very little in trials targeting conditions known to persistently underrepresent women (e.g., cardiovascular disease, sex-neutral cancers). Finally, only one article made the explicit distinction between sex and gender among participants identified as women.

Prior work on trial recruitment provides context for our findings albeit through a sex and gender-neutral lens. In an umbrella review by Rodriguez-Torrez et al. [26], barriers and facilitators to trial participation were described across 20 distinct themes. Most relevant included concerns about intervention characteristics (e.g., risk of side effects or time burden), personal obstacles to participation (e.g., transportation, childcare, work schedules), need for information about the trial (e.g., clarity and messaging about trial participation requirements), and the influence of others on decisions to participate (e.g., friends, family, institutions). While their analysis was not stratified by sex or gender, the barriers noted by Rodriguez-Torrez and colleagues are typically more common among women due to the typical gendered distribution of social responsibilities related to childcare and reliance on social support for decision making. Many of the strategies that we identified were directly relevant to these established barriers (e.g., compensation for childcare, relationship management during trial interactions).

A second prior review by Treweek et al. [13] reported on 68 eligible trials exploring different design features and their impacts on trial recruitment, though they also did not include a sex-specific analysis. They noted 3 design features whose impact was found to have a high certainty of evidence, including open vs. blinded/placebo trials, telephone reminders to those who do not respond to postal invitations, and using a bespoke approach to developing participant information materials drawing on population-specific input. While telephone reminders are a popular recruitment approach for targeting underrepresented populations, as illustrated by its appearance in 38 of our articles reviewed, this approach was found to be ineffective by Treweek et al. It is possible that population-specific tailoring of study materials is more effective among certain groups. For example, women Veterans were more likely to participate in a survey study when approached using “enhanced recruitment materials” designed to build trust through pictures of the study team and detailed information about their skills and experience [27].

There is also overlap with our findings among study teams aiming to enhance the participation of other specific patient populations. Bonevski et al. [28] conducted a review of recruitment strategies targeting socially disadvantaged populations. They describe many similar strategies to those we identified including population-tailoring of study materials, use of alternative sampling strategies (e.g., snowball, targeted, oversampling); community-engaged approaches such as community outreach, patient collaborators input on intervention design research and recruitment, incentives; flexibility in data collection (e.g., timing and/or modality); and use of bilingual materials. Obtaining the endorsement of community leaders and engagement of community members to provide culturally congruent expertise has also been employed to boost population-specific representation [29].

One challenge to enhancing the inclusion of women in trials is that women as a population are not a homogenous community. An individual who identifies as a woman may or may not have been assigned ‘female’ at birth (i.e., Cis-gender) and will bring her own intersectional collection of identities to the research setting. We found no studies that identified participants by both sex and gender nor any that specified recruitment goals across these constructs. Collecting and reporting sex and gender identity is critical to support future work to understanding how recruitment approaches might be tailored to ensure sex and gender parity in trial participation. Fortunately, many of the identified strategies that promote flexibility and ease of participation will likely benefit all potential participants and could be applied as universal design principles. For example, while potentially more common among women, barriers such as caregiving responsibilities [30], unemployment, and transportation problems [31], could be addressed by identified strategies such as flexible intervention delivery modality and minimal data collection burden.

Women are as likely, or more likely, to participate in research as men when given the opportunity [32,33,34] which may explain why the majority of strategies identified in this review were around potential participant outreach. However, study retention also warrants attention. Prior work on retention among women in research has identified ongoing remote contact as a helpful strategy [35], along with the importance of interactions with study staff [28, 31]. Purposeful trust-building and establishing a positive-caring rapport within participant-study staff relationships is an important strategy for other historically marginalized populations within research and minoritized populations are often willing to participate despite underlying distrust [36, 37]. Interestingly, we found relatively little around the specific training of study staff or the establishment and promotion of the trial staff-participant relationship. Only one study described sex and/or gender awareness training, four reported race/ethnicity-focused training, and only 2 noted intentional relationship building with participants. Other approaches to staff training could include engendering a trust-worthy study environment through the incorporation of trauma-informed care principles (TIC). TIC was first developed within the context of mental health treatment as an approach to normalize the individual reaction to a traumatic experience and focus on a strengths-based approach to recovery [38]. TIC is now being incorporated into multiple clinical settings including virtual primary care [39, 40].

In addition to being the first review to focus solely on strategies employed to boost the representation of women, we also considered strategies across the entire lifespan of a clinical trial. Our approach adds to that of Bonevski et al. who expanded their perspective on trial phases beyond outreach and trial design but did not consider analysis or dissemination. Accordingly, the next steps in the inclusion of women in clinical trials will need to involve a comprehensive and a priori-defined approach to the deployment of strategies across the lifespan of the trial. Work from the Collaborative Institutional Training Program (CITI) has emphasized the importance of “upstream” recruitment planning and incorporated this approach into their recruitment framework [41]. In fact, among the most effective efforts to boost inclusion in the National Lung Screening Trial were setting an a priori recruitment goal for individuals from minoritized populations and planning trial recruitment efforts in advance [42]. An example of a population-specific tool for recruitment and retention can be found in the “5Ts” framework for the inclusion of older adults [43] which outlines key steps to ensuring that clinical studies are accommodating to the needs of older adults (e.g., allowing more time and tips to accommodate).

Strengths of this work include using a standardized, rigorous, and a priori-defined protocol; however, our findings should be considered within the context of its limitations. First, we approached this review as an evidence map to describe the breadth of literature related to the topic of enhancing the representation of women in trials; therefore, in keeping with this review approach, we did not conduct a quality assessment of the identified literature or propose specific conclusions about the most impactful strategies for the inclusion of women. However, we have included the author’s reported conclusions in an additional file as a reference for future analysis (see Additional file 9). We note that similar to work with other targeted populations [28, 44], there were few trials directly comparing strategies which will limit the drawing of inferences about strategy effectiveness. This reality reflects the dearth of efficacious evidence-driven approaches to boost gender diversity in recruitment, further signaling a need for higher-level exploration of the differences between genders and sexes [42]. A second limitation was the variability in the definition of “women” among authors, which often conflated “gender” and “sex,” making it difficult to accurately identify approaches boosting gender diversity among recruitment. In addition, we did not evaluate if the strategies described would be applicable to gender-diverse persons as well as who may be susceptible to sex-specific conditions (e.g., ovarian cancer). This blurs the important distinction between sex and gender which is an important area for future work. When possible, we identified when a single study was described across multiple articles; however, it is possible that we missed some such studies due to a lack of reported detail in the included articles. Finally, we describe strategies as written by the investigators from individual trials. We suspect that there were likely actions taken to enhance the representation of women but which were not documented. In particular, this is possibly related to analysis and dissemination that may have been mentioned in main results articles from relevant trials but not included in manuscripts related to recruitment and retention if not conceptualized as relevant.

Conclusion

The research community has called for equitable representation of women in clinical trials to ensure the generalizability of scientific evidence and to inform sex-specific, evidence-based care. Strategies to enhance the representation of women in clinical research need to be considered across the lifespan of a trial to promote long-term participant engagement stemming from investment and trust-building beyond the initial signing of a consent form. While many such strategies have been used to date, very few have been described in the context of non-sex-specific research (e.g., cardiovascular disease, cancer). The persistent representation gap in these critical areas of clinical research limits their generalizability and stands in the way of patient-centered, evidence-based care for women. Future work should explore which strategies are most effective to ensure the appropriate participation of women in clinical research on conditions relevant to both men and women.