Abstract
Background
This study aims to review the literature regarding the barriers to sampling, recruitment, participation, and retention of members of socioeconomically disadvantaged groups in health research and strategies for increasing the amount of health research conducted with socially disadvantaged groups.
Methods
A systematic review with narrative synthesis was conducted. Searches of electronic databases Medline, PsychInfo, EMBASE, Social Science Index via Web of Knowledge and CINHAL were conducted for English language articles published up to May 2013. Qualitative and quantitative studies as well as literature reviews were included. Articles were included if they reported attempts to increase disadvantaged group participation in research, or the barriers to research with disadvantaged groups. Groups of interest were those described as socially, culturally or financially disadvantaged compared to the majority of society. Eligible articles were categorised according to five phases of research: 1) sampling, 2) recruitment and gaining consent, 3) data collection and measurement, 4) intervention delivery and uptake, and 5) retention and attrition.
Results
In total, 116 papers from 115 studies met inclusion criteria and 31 previous literature reviews were included. A comprehensive summation of the major barriers to working with various disadvantaged groups is provided, along with proposed strategies for addressing each of the identified types of barriers. Most studies of strategies to address the barriers were of a descriptive nature and only nine studies reported the results of randomised trials.
Conclusions
To tackle the challenges of research with socially disadvantaged groups, and increase their representation in health and medical research, researchers and research institutions need to acknowledge extended timeframes, plan for higher resourcing costs and operate via community partnerships.
Similar content being viewed by others
Background
The omission of groups of lower socioeconomic status from public health and medical research has been observed for some time regardless of type of research study [1]. In most Western developed countries white, middle class, highly educated males tend to be overrepresented in health and medical research and people from socially disadvantaged groups under-represented [1, 2].
Failure to obtain medical research data which accurately reflects the breadth of the whole population poses a number of drawbacks including threats to external validity and ability to generalise [3], denying excluded groups from any health benefits of trial participation [4], inability to check the safety of health innovations with sub-groups in the population [5], and failing to identify groups that have the highest burden of illness and developing an understanding of why differences exist [6].
Researchers continue to struggle to access, engage and retain participants from socially disadvantaged groups [7], resulting in labels such as “hard-to-reach” or “hidden”. According to Sydor’s [8] definition that “hard to reach populations are difficult for researchers to access”, and Lambert and Wiebel’s [9] definition of hidden populations as “those who are disadvantaged and disenfranchised: the homeless and transient, chronically mentally ill, high school drop-outs, criminal offenders, prostitutes, juvenile delinquents, gang members, runaways and other street people”, socially disadvantaged groups are difficult for researchers to access cost-efficiently in large numbers necessary for statistically powerful study designs. There are many reasons why socially disadvantaged groups are not included in health and medical research. Understanding these factors is necessary for developing strategies to increase the level of involvement and participation in health and medical research for disadvantaged groups. This study aims to review the literature regarding the barriers to sampling, recruitment, participation, and retention of members of socially disadvantaged groups in health research and the strategies for overcoming the barriers that may help increase the amount of health research conducted with socially disadvantaged groups.
Methods
Search strategy
Searches of electronic databases Medline, PsychInfo, EMBASE, Social Science Index via Web of Knowledge and CINHAL were conducted for English language articles published up to May 2013. A wide-net search strategy involving combinations of the following keywords was initially conducted to capture as broad a sample of studies as possible: “difficult to reach” or “hard to reach” or “social disadvantage” and “health research”. An iterative process was used where more general searches were conducted initially, with papers identified informing subsequent targeted searches. A general internet search with Google Scholar also assisted in the search for grey literature. Free text searching was implemented using the following key words: ‘hard-to-reach’ , ‘difficult-to-reach’ , and ‘disadvantaged’ with ‘health’ and ‘health research’. In addition, manual checks of the reference lists of retrieved articles and citation searches were conducted.
Selection criteria
Articles were included if they reported attempts to increase socially disadvantaged group participation in research, or barriers to conducting research with socially disadvantaged groups. Socially disadvantaged groups were defined as socially, culturally or financially disadvantaged compared to the majority of society, implying individual, environmental or social restrictions to their opportunities to participate in health research [8–10]. In order to capture a broad representation of evidence, qualitative, quantitative, mixed methods, case studies and literature reviews were included. Only studies that focused on health-related research were included.
Studies were excluded if they: described improving access for socially disadvantaged groups to health services rather than health research; were primary articles which had been cited in the previous literature reviews; focused on age (e.g., adolescents) or gender groups (e.g., females) without explicitly stating that the group was socially disadvantaged. Editorials and commentaries were also excluded.
Data extraction
Articles obtained from the electronic database searches were assessed by one reviewer (MR) in two phases: 1) title and abstract review, and if it appeared relevant, 2) full text review.
Together with a second reviewer (BB) full text review of articles obtained was conducted. Uncertainty was resolved through consultation with the other reviewer. Two reviewers (BB and MR) extracted data on country, descriptors of the study’s target sample group, variables measured, study design, and key findings on a) the barriers to participation in health research; and b) strategies to improve participation were extracted and summarised.
The results of the included studies were further categorised according to five stages of a research study where representativeness may be threatened: 1) developing a sampling frame; 2) recruitment and gaining consent; 3) data collection and measurement; 4) intervention delivery and uptake (if applicable); and 5) retention and attrition. These stages were imposed by the study authors prior to data extraction.
Assessment of risk of bias
Given the considerable heterogeneity of study types, a systematic scoring system for evaluating the methodological characteristics of individual studies or risk of bias was not applied. Instead, a graded system of ‘levels of evidence’ based on study design alone was used. A number of hierarchies of evidence exist [11–13] and were adapted for this study (refer to Table 1).
Results
The electronic searches yielded 8,497 potential articles and 36 articles were identified using additional search strategies. Following removal of duplicates, eligibility screening yielded 116 primary source papers from 115 studies (one study had two publications [14, 15]) that met inclusion criteria (see Figure 1).
Flow chart of included studies. Footnote: a Did not report barriers or strategies to improve participation (n = 3118); Not socially disadvantaged group (n = 453); Focus on health service participation rather than health research participation (n = 91); Not English language (n = 58). b Focus on health service participation rather than health research participation (n = 53); Not socially disadvantaged group (n = 39).
The majority of the included studies (n = 76) were based on research conducted in the United States of America (US) [16–91], 23 papers describing 22 studies were conducted in the United Kingdom (UK) [14, 15, 92–112], nine in Australia [113–121], four in Canada [122–125], two in Europe (Russia and Estonia [126] and Germany [127]), one in New Zealand [128], and one US-led study based in Mexico [129]. Table 2 summarises the socially disadvantaged groups targeted in the included studies. African American and diverse ethnic and racial groups (dominated by US based studies) were the most studied groups.
The vast majority of articles describing strategies for improving representativeness were non-experimental descriptive surveys, qualitative studies or case studies (see Table 1). Only 7 articles reported the results of randomised controlled trials.
In addition, 31 previous reviews of related literature were identified and summarised in Table 3[130–158] including two papers which presented both original data as well as a literature review [14, 34].
Developing a sampling frame
Barriers
Of the 31 literature reviews, 10 focused on sampling [34, 136–138, 149–154] and 33 primary source articles on sampling issues were found [32–34, 38, 44, 47, 51, 53, 54, 57],[60–62, 67, 74, 77–79, 81, 84, 91, 95],[97, 102–104, 113, 116, 118, 122, 126, 128] (see Table 4). Random population samples are often insufficient to accumulate large enough samples of hard-to-reach groups.
This barrier was reported across study types, from large scale population health surveys to intervention trials. Some studies identified difficulties sampling groups defined as “hidden populations” consisting mostly of people who do not want to be identified such as people who use illegal substances or self-identified groups such as homosexual people, others described difficulties sampling groups with low numbers in the population such as Indigenous people. As a result population based probability sampling tends to be a time and cost inefficient strategy for sampling socially disadvantaged groups.
Strategies to improve sampling
Non-probability sampling
A number of alternatives to random probability sampling were described (see Table 4) including snowball/social network or respondent-driven recruitment, venue based time- location sampling, targeted sampling, capture-recapture, adaptive sampling and oversampling of low prevalence population sub-groups [33, 34, 38, 53, 57, 79, 84, 95],[102, 104, 113, 126, 129].
Except for oversampling methods for low prevalence populations [91, 149], all of the other sampling strategies require formative research to identify venues (places), times, and contact persons to develop a targeted sampling frame for the group of interest which may impose significant time and cost to the research. Some strategies such as snowball and respondent-driven sampling involve referral chains of sampling. Selection bias and gatekeeper bias which limit validity of the sample are the primary limitations of these strategies. While these issues may not be problematic for studies which do not require representativeness for generalizability, such as qualitative research, they do have limited use in quantitative research.
Sampling through community organisations
One option for creating a sampling frame for specific socially disadvantaged groups is to collaborate with community organisations with access to those groups and to draw a convenience sample through that organisation [44, 47, 54, 60–62, 67, 74, 103, 116],[118, 122, 128, 136]. Benoit et al. [122] variously defined community group as any group with high access to the target population and partnerships have taken three main forms: a) the community group helping researchers gain access to an otherwise hard-to-reach group; b) a reciprocal relationship in which community members and researchers have knowledge and learn from the other; and c) community-initiated research projects that seek academic partnerships and use the outcomes to direct policy and program delivery. While this form of convenience sampling may not be representative of the general target group, it presents pragmatic advantages for sampling large numbers of members of socially disadvantaged groups.
Combinations of sampling strategies
Fifteen studies described the use of a combination of sampling strategies [32, 34, 44, 47, 54, 60, 61, 67],[74, 103, 104, 113, 116, 122, 128]. For example, Shedlin et al. [60] used snowball sampling within community groups. As most reports were case-studies it is not possible to compare the effectiveness of combined approaches.
Comparisons of strategies
A number of studies provided comparisons of different sampling strategies such as respondent driven sampling compared with targeted sampling finding that in most cases these sampling approaches produced similar sample size and representativeness, however there were differences in costs. Platt et al. [126] compared respondent driven sampling with snowball sampling to reach high risk HIV participants and found that although snowball sampling was more costly, it resulted in greater participant numbers. Keyzer et al. [62] found that direct mail, community outreach (including presentations and visits to churches and community centres) and recruitment through a health education council were the most cost-effective strategies for recruiting minority groups, while use of mass media and advertising was a high-cost low-yield strategy. In New Zealand, Mhurchu et al. [128] trialled three strategies for capturing Maori and Pacific Islander participants in their research of a tailored nutrition intervention on supermarket purchases; a mail-out to supermarket customers, approaching Maori and Pacific community groups located close to the supermarkets and ‘in-store’ recruitment where recruiters approached Maori and Pacific customers as they entered the supermarket. While the mail-out resulted in the highest overall number of participants, only 11% were Maori or Pacific Islander. In comparison, community and in-store strategies recruited the lowest numbers overall, but 96% of participants were Maori or Pacific Islander.
Other strategies
Dowrick et al. [97] suggest use of multiple sources of secondary and primary data, such as previous published data and qualitative interviews, as a strategy to overcome sampling difficulties in a study to assess mental health needs of hard to reach groups. This approach has limited applicability and cannot be used for prevalence surveys or intervention trials where contact with individual participants is required. Mathews & Cramer [137] suggested the use of the internet to identify and sample the “hidden” population of gay, lesbian, bisexual and transgender (GLBT) and Mathy et al. [51] described sampling methods over the internet for quantitative and qualitative research. No empirical evidence of effectiveness is presented.
Recruitment and gaining consent
Barriers
Most of the literature reviews (20/31) discussed problems and solutions to recruiting socially disadvantaged populations into health research [14, 15, 131, 132, 134–148, 156, 158]. In addition, 58 primary source articles considered issues relating to low recruitment rates [16, 18, 21, 22, 26, 28, 30, 31],[35–37, 40–43, 45, 46, 50, 51, 55, 56],[60, 64, 65, 70–73, 75, 77, 78, 80–83],[85, 86, 89, 92–94, 96, 99–101, 105, 106],[109, 112–115, 119, 123–125, 127] (see Table 5).
Reasons provided for low response rates in research with socially disadvantaged groups included: mistrust in research or researchers, particularly amongst African Americans and Indigenous populations who had a history of being mistreated in medical research; fear of authority; and perceptions that participation presented no personal benefit to them or their community and may cause potential harm, stigma, mistreatment or exploitation. Similarly members of some groups may fail to participate in research out of fear of being publically exposed, particularly if they engaged in illegal behaviours such as prostitution, gambling or illicit drug use or are socially stigmatised, such as people with HIV or AIDS or people who are GLBT. One review however found that willingness to participate in research was as high in racial and ethnic minority groups as it was in Caucasian participants [146], suggesting that factors other than participant attitudes or beliefs play a role in limiting health research participation.
Other barriers to the recruitment of vulnerable populations included cultural beliefs prohibiting participation; age and gender issues whereby in some cultures discussing health issues is viewed as ‘sensitive’ , particularly if female, elderly or young; and a lack of awareness of health research or education about participation in health research. Low literacy affecting ability to provide informed consent was also described as a barrier to recruitment.
Gatekeepers who restrict access to health research have been identified by some studies as a barrier for research participation. Some health professionals, who have the opportunity to encourage research participation have been found to fail to do so due to paternalistic beliefs that people in lower socioeconomic groups don’t have the time, interest or ability to participate, or have poor communication skills.
Participant lack of understanding of the research information, process or significance was reported as barriers to gaining consent. In one case, participants not understanding the need for “random” sampling was reported as a barrier to gaining consent, since potential participants believed that those who needed the research should be approached, not randomly sampled [113]. Other papers highlighted the restrictive nature of some eligibility criteria (such as language or comorbidity-related restrictions) which excluded socially disadvantaged groups, particularly in clinical trials [14, 15, 50].
Improving response rates
Community-research partnerships
In order to address some of the recruitment barriers relating to mistrust or fear of research, and gatekeepers impeding recruitment, 25 studies [18, 21, 22, 31, 40, 55, 56, 60],[63, 70, 77, 78, 80, 82, 93, 94],[96, 100, 101, 105, 113–115, 124, 125] and eight reviews [131, 135, 136, 141, 144, 145] suggested that community groups be involved in the research and recruitment process. This may be particularly effective for communities that have hierarchical structures such as Australian Aboriginal communities who look to their elders to provide leadership. Some reports have suggested that engagement of local peer or known community members as ‘recruiters’ will increase trust and response rates [21, 40, 55, 82, 96, 105, 121, 123],[124]. Similarly, the use of community advisory groups is likely to be beneficial and increase the perception that the research is community-driven and responsive [28, 47, 100, 113, 118, 135]. Germino et al. [28] used a comprehensive community based approach to recruit a representative sample of African American cancer survivors. The approach was designed to address recruitment barriers of mistrust and enhance familiarity.
They engaged a number of community groups, for promotion and education about the research including ‘cultural brokers’ to liaise between participants and researchers. They reported high recruitment and retention rates.
These community based strategies offer shared ownership of the data and publications produced as a result of the research; ensuring that the research will provide either sustainable programs beyond the life of the research project, or links to services and resources. Providing gifts, financial incentives or thank you awards and ceremonies which include feedback to the community about the outcomes of the research were also presented as important components of community-based recruitment [18, 30, 92–94, 127, 158].
In a review of recruitment strategies for clinical trials with minority groups by UyBico et al. [141], community organisation-based recruitment was found to be the least effective form when compared to social marketing, use of health services and referral based recruitment. Similarly, Martin et al. [21] found that despite extensive community consultation and use of community based facilities and bilingual recruitment materials, low recruitment of Mexican American participants persisted. Recruitment increased once local Spanish speaking workers were engaged to conduct outreach recruitment.
Use of media and social marketing
Eleven studies reported the use of media and social marketing techniques tailored to the target audience [18, 45, 46, 65, 72, 73, 81, 85],[86, 99, 119]. However, these studies fail to provide a guide as to which medium would be most effective with different target groups and instead suggest that formative research should determine the most appropriate strategy. UyBico et al’s [141] review found social marketing (defined as mass media, mass mailings and mass telephone calls) to be the most effective recruitment strategy for minority groups into clinical trials, compared with health provider recruitment and community organisation-based recruitment.
A key component which is common across studies recommending the use of different recruitment channels is the need for the recruitment channel to be culturally and linguistically appropriate [16, 45, 71, 86, 92]. This can address both barriers of lack of awareness of health research, and cultural barriers. Studies also highlight the need to educate research staff to ensure cultural competencies and understanding which are likely to enhance response rates [22, 30].
Strategies to encourage gatekeeper support
Strategies for addressing the barrier of gatekeepers include employing gatekeepers as project recruitment officers and involving them in the research [22, 93, 100, 113, 118]; ensuring that community authorities are informed about the research and adequately consulted [113, 114]; and paying health professionals through financial incentives to assist with recruitment [125, 143]. Loftin et al. [71] reported slow recruitment and low response rates in a study using primary health care provider identification and recruitment of African Americans into diabetes research. One review suggested developing patient materials that can be distributed directly to potential participants of clinical trials thus overcoming the barriers of clinicians being too busy to recruit or having poor communication skills [143]. Mathy et al. [51] compared an internet derived sample with a Gallup poll sample of the US general population. They found the samples equivalent in educational distribution and geographical location (rural and urban), and the internet sample reached more representatives of lower income and ethnic diversity.
Comparison studies
Four of the nine RCTs examined the effectiveness of recruitment strategies [36, 42, 43, 106], as did three non-RCT comparison studies [41, 75, 83]. In a RCT by Satia et al. [43], potential African American participants were randomly assigned to receive either generic or culturally sensitive invitation letters. Within each letter group participants were randomly assigned to receive a small incentive (a telephone card worth USD $3.60). While the overall response rate was low (17.5%), it was significantly higher for those receiving the incentive (23.9%) compared with those not receiving an incentive (15.8%). There was no difference in response rates between those receiving the generic invitation letter and the ‘culturally sensitive’ letter [43]. Maxwell et al. [42] trialled three incentive conditions (no incentive, USD $5 cash or $20 promise upon completion) to increase response rates to a baseline survey amongst racially diverse groups (Latino, Asian and African American). No differences were found and response rates were low (28%-37%). Another RCT assessed whether an advance letter mailed out two weeks prior to a mailed survey would enhance response rates among African American participants compared with White American participants [36]. Statistically significant differences were found for white American participants only illustrating how a strategy can potentially contribute to disparities. In a RCT of hand-written envelopes compared with printed envelopes to increase survey response rates in a socioeconomically disadvantaged area, Choudhury et al. [106] found that the hand-written envelopes resulted in slightly but not significantly higher response rate (17% vs. 14%). Using ‘debriefing questionnaires’ to glean interviewer experiences of recruiting subjects using three different methods, McLean & Campbell [109] found that local advertisements and use of media recruited the highest number of white English participants, interpersonal contacts increased recruitment of Pakistani-Kashmiri subjects and organisational contacts recruited the most African-Caribbean subjects. All approaches included financial incentives for participation. Oakley et al. [112] compared the cost of using interpreters to increase recruitment of non-English speaking women. Costs per person recruited were higher for women who needed an interpreter (average of £135) compared with for those who did not (£80).
One unique study [25] sought to develop an African American ‘health research registry’ for easier future recruitment and engagement with research. The study compared direct recruitment methods (using existing study sample databases; public databases; community outreach) with indirect methods (radio, internet and email). The study concluded that all strategies were needed to recruit a representative sample into the registry because some strategies were more successful with sub-groups (e.g., email and internet methods recruited younger participants).
Other strategies
A number of reports have suggested that study inclusion criteria need to be broadened in order to increase the representativeness of samples, particularly in relation to comorbidities [35, 50]. Adams-Campbell [50] suggested that investigators incorporate examination of comorbidities into clinical trial study protocols.
A novel suggestion to tackle the barrier of uncertainty about ‘random’ sampling is to include others within the participants’ immediate family or community in data collection [113, 135], but only to use the data from the individual that was randomly sampled. Clearly this approach has ethical, resource and cost implications.
Sutherland et al. [26] described a range of personalised strategies to aid recruitment of abused rural women into sensitive research about the risk of sexually transmitted disease.
They found that personalised approaches like knitting together, and changing terminology to promote trusting relationships (such as “the nurses doing the research” instead of “the researchers”) boosted recruitment rates [26].
Data collection and measurement
Barriers
In total, 29 studies [45, 48, 49, 52, 54–56, 59, 60, 63],[68, 74, 85, 90, 96–98, 100, 105, 106],[110, 111, 113–115, 117, 118, 120, 124] and seven reviews [133–138, 140] explored barriers and solutions for collecting research data with participants from socially disadvantaged groups (see Table 6).
The need for complete and representative data from health research studies is common across study types but is particularly important in surveys. Language, lack of education and low literacy barriers may prohibit the collection of certain types of data such as self-administered survey data. Similarly, a barrier which is specific to telephone-based data collection is the lack of landline telephones amongst disadvantaged groups, limiting their inclusion in epidemiological and population-based research. Shebl et al. [49], for example found significant differences between those with or without landline telephones in race/ethnicity, health care access, insurance coverage and several types of health behaviours including smoking status and cancer screening behaviours. The resulting bias suggests that population-wide surveys utilising only landline telephone surveying techniques are not providing accurate estimates of health behaviours. Finally suspicion regarding the use of the data collected and mistrust of researchers were identified as barriers.
Improving data collection and measurement methods
Inclusive language and methods
One of the most common strategies to improve inclusion of linguistically diverse or low literacy groups in health research is to simplify the reading age of the study materials or to translate materials into other more common languages [45, 63, 96, 113] and use bilingual research assistants [45, 85, 90, 100, 111]. A number of studies highlighted the need for culturally trained and skilled field-workers [56, 111, 140] or employing locals or peers to conduct field work [55, 74, 96, 98, 105, 115, 118]. Use of “insiders” (peer or local researchers) offers the added advantage of addressing any researcher mistrust or suspicion [105] as well as building the capacity of the community or organisation in conducting research. One method of data collection called Photovoice allows participants to use photos and pictures to respond to spoken questions or scenarios and to tell a ‘story’ [133]. Photovoice has been used with Australian Aboriginal communities where the telling of stories is often through paintings and art. Its use is limited as an exploratory qualitative tool and does not provide large-scale quantitative data.
Flexible data collection methods
In order to address barriers to socially disadvantaged groups participating in telephone surveys, a number of studies have outlined the need for flexibility and tailoring of data collection methods to participant circumstances [60, 113, 114]. If participants cannot be reached by telephone data collection, telephone surveys could be supplemented by face-to-face door knock interviews [49] or online surveys [52]. Allison et al. [110] caution that supplementing postal questionnaires with face to face interviews may not produce responses with equivalence as comparisons of these two modes of survey delivery resulted in very low kappa agreement scores for some items. Working with Australian Aboriginal communities Couzos et al. [114] proposed a ‘community-controlled’ research process including data collection and management which was flexible and involved locals employed to administer the research protocol.
Thomas et al. [117] has suggested that self-administered surveys should be avoided if possible with disadvantaged groups where it is difficult to collect data directly from the individual, and instead, community-wide objective data be used. These authors give the example of tobacco and alcohol sales data from Australian Aboriginal communities. This measure is only a proxy to behavioural measures and provides limited data about how the tobacco or alcohol might have been used and by whom. However, in some cases the objective data could be supplemented with smaller targeted qualitative research using methods such as Photovoice ([133]: see above) to gain a more complete set of information about a health concern or health behaviour. Similarly, Dowrick et al. [97] suggest using secondary data sources supplemented with small qualitative data collection as measurement tools in studies assessing health service needs with socially disadvantaged groups.
Use of technology to gather data
Five studies suggested using the internet or other technology-based strategies to collect data from socially disadvantaged groups such as those living in rural areas [48, 52, 54, 68, 120].
Hahn and Cella [48] described the acceptability of a touchscreen computer delivered health survey with patients categorised with high or low literacy ability and found that acceptability was high across both groups. Almost all patients (98%) felt the touchscreen survey was easy or very easy to use.
Pilot testing measures
Hing et al. [115] and Couzos et al. [114] highlight the importance of involving local community partners in the development of research protocols and materials when working with Australian Indigenous communities and the importance of extensive pilot testing of materials [100].
Comparison studies
One randomised trial of survey methods was included in the review [90]. In this trial, Ngo-Metzer et al. [90] compared data quality (response rates, missing data, internal consistency reliability and non-response bias) following a mailed health survey compared to one delivered by telephone with Asian American participants. The surveys were provided in both English and the target language (either printed or through bilingual interviews). No differences in reliability or non-response bias were found. The telephone interview resulted in a significantly higher response rate (75% versus 59%) and fewer missing items (1.67 versus 4.14) [90]. In a pre- and post-comparison study, Choudhury et al. [106] found that shortening a 12-page questionnaire to two pages significantly increased the response rate (37% versus 12%) in a study with respondents from economically deprived multicultural areas. For assessing substance abuse, Alemagno et al. [59] compared a telephone-based interactive voice response system to a face to face interview and found high reliability (% agreement = 80-95% and test-retest kappa = 0.65 – 0.91) and validity of the computer assisted telephone method (78% agreement with biochemical measure of substance use). The computerised voice telephone system offered participants anonymity [59].
Intervention delivery and uptake
Barriers
Thirty six articles [19, 20, 27, 29, 31, 35, 37, 40],[52, 55, 56, 58, 64, 67–69, 71, 73],[77, 78, 80, 82, 85, 87, 88, 105],[107, 108, 114, 118, 122, 124] and five reviews [130–132, 148, 157] considered intervention fidelity (see Table 7).
Concerns about randomisation - that some members of the community will not receive what may be a beneficial intervention - has threatened the implementation of intervention trials in some communities [40, 148]. In some cases the concern has been about the loss of control over deciding who receives an intervention and who does not [132]. These threats could result in contamination if intervention group participants or those conducting the randomisation offer the intervention to control group participants.
Equally of concern are interventions that do not align with the perceived needs or priorities of the target group or interventions (in terms of both content and delivery) which are not acceptable, feasible and culturally appropriate to community values and beliefs [71, 80].
Improving intervention participation and fidelity
Alternative methodologies and study designs
Alternative study designs to the classic RCT have been proposed that may be more acceptable [27, 131] such as multiple baseline designs, stepped wedge designs and wait-list control groups where the intervention is delivered to all groups at different times [29].
Yancey [148] outlines other designs whereby the control group either receive an alternative treatment or they receive the intervention after the trial is over (a waitlist control). Similarly, Hough et al. [40] randomised homeless people to four conditions; each providing interventions that were greater than the services participants were receiving before the study. Yancey et al. [148] and Woods et al. [73] also highlight the importance of educating participants of the need for randomisation to enhance understanding and ensuring that the control group receive, at the very least, usual care and that they are not having health care denied them.
Community/participant involvement in intervention design
Seventeen studies [27, 31, 35, 56, 67, 70, 71, 74],[77, 80, 85, 107, 108, 113, 114, 118],[122] suggested community involvement in the design of health interventions as a means of avoiding the barrier of culturally inappropriate interventions. While the opportunity to design pharmacological and clinical medical interventions together with potential participants is very limited, it is a strategy that is particularly relevant to public health or behavioural interventions. Involving the target group in intervention design for public health interventions and careful inclusive use of formative research to ensure that both the content and delivery of the intervention is acceptable to the target group can increase likelihood of uptake [70, 80, 113, 114, 118].
To design culturally appropriate smoking cessation interventions for American Indian communities, Fu et al. [29] conducted focus groups. Participants reported that the following features of the intervention were important to them: programs led by American Indians, opportunity to link with other American Indians interested in quitting, free nicotine replacement therapy, incentives, and culturally specific program components such as American Indian images, education on traditional tobacco use, messages that value family and include narratives or stories [29].
Ammerman et al. [31] described the development of a culturally appropriate intervention which was based on theories of sustainability and diffusion. This is viewed as an attempt to “give back” to participating communities through sustainable interventions which are adopted beyond the life of the research [31, 78, 114] and which build capacity within communities to address the health issue independently of the research [56, 124].
Modifying interventions implemented in general populations so they are better tailored to disadvantaged groups such as low literacy or rural groups [19, 52, 58, 68, 69, 87, 130, 157] has been suggested. Other studies emphasise the benefits of including community members, locals or peers as intervention delivery agents for improving compliance to the intervention [20, 55, 58, 82, 105, 108, 124, 157]. For example, Hughes et al. [82] outlines a peer educator participant-driven intervention for injecting drug users as an ethical public health model. Similarly, Rothschild et al. [20] employed bilingual Mexican American Community Health Workers from local neighbourhoods as culturally competent peer interventionists in their trial of a diabetes management intervention for Mexican Americans and reported high intervention participation and fidelity.
Comparison studies
Two RCTs of intervention strategies tailored for disadvantaged groups were included in the review. In a blinded randomised trial, Nicholson et al. [37] compared emotional and behavioural responses to four versions of an information intervention provided to Africa-American communities regarding colorectal cancer screening (1. Emphasising impact on African-Americans, 2. “Blacks are doing worse than whites”, 3. “Blacks are improving but less than whites”, and 4. Progress – “Blacks are improving over time”). Participants exposed to the two versions highlighting disparities framed in a negative way (2 and 3 above), reported more negative emotional reactions. In contrast the progress framing (4 above), elicited more positive responses and potential participants were more likely to agree to be screened. The results of this trial suggest that the way in which health information is presented can influence attitudes and intentions, with reports about progress yielding a more positive effect on intention. The authors note that this is especially important among those with high levels of medical mistrust. Befort et al. [88] compared the effectiveness of group versus one-on-one telephone counselling for 34 women in hard-to-reach rural areas.
Compliance with the 24-week program was similar between groups, although they did report a significant improvement in weight loss with those in the group-based intervention losing more than those receiving individual counselling. Befort et al. [88] also found the group program to be cost-effective (USD714.43 versus USD1029.06).
Other
In a systematic review of diabetes care interventions for socially disadvantaged groups, Glazier et al. [157] noted that the following factors improved both health outcomes, and participation with the intervention: one-on-one interventions, focussing on behaviour-related tasks, providing feedback, and high intensity interventions delivered over a long duration. The review found that interventions which used mainly didactic teaching and focussed on diabetes knowledge were the least successful.
Attrition and retention
Barriers
Twenty six studies [16–18, 20, 22–24, 30, 35, 39, 40, 54],[65, 66, 70–76, 80, 81, 85, 96, 123] and six reviews [135, 139, 145, 147, 148, 158] examined barriers to retaining participants in research and strategies for maximising retention rates. The most common barrier reported related to follow-up data collection was difficulty maintaining participant contact. Challenges to maintaining contact included the transient nature of the lives of those in many socioeconomically disadvantaged groups, with phone numbers and addresses changing frequently. Practical barriers often related to their socially disadvantaged status such as transport difficulties, inability to take time away from work to participate, lack of child-care or simply forgetting about the research which is competing with other priorities of daily living [73].
Improving retention rates
Incentives and gifts
The most commonly trialled strategy for maintaining involvement of participants throughout a research project was the use of incentives and gifts [18, 22–24, 30, 39, 40, 71–73, 76, 80],[81, 123, 158]. Cash incentives have been found to be more effective than non-cash incentives [23], however the use of study branding or logos on non-cash gifts has been reported to be effective in case studies at keeping participants involved [74]. In a study with young Latino women, Lindenberg et al. [80] found that financial cash incentives were considered unacceptable and that vouchers for grocery or department stores were preferred.
Other studies attempted to maintain contact and participations with other gifts such as clothing, birthday and other holiday cards, mugs and personalised follow-up letters [71, 73, 80], coffee, food or drinks [40]. If travel is required, Woods et al. [73] found that focus group participants suggested transport vouchers and bus tokens as incentives to return to the study [71, 73]. Some researchers have noted that these little gifts build trust and relationships between participants and researchers [40, 70].
Another two studies of patients receiving treatment for substance abuse found that cash payments did not lead to use of the incentive to purchase illicit drugs and participants reported no perception of coercion [23, 24]. In fact the studies found high participant satisfaction with the study, better follow-up rates and reduced tracking efforts.
Multiple tracking, reminders and contact procedures
Keeping in regular contact with participants has been reported as an effective method of lowering attrition. Ensuring that researchers have multiple forms of contact for each participant (e.g., phone, mail, email, address and other contact persons), has been reported to be essential for maintaining contact with participants from vulnerable groups who may be highly mobile [17, 20, 22, 30, 39, 40, 54, 75],[81, 85] as well as having contact details of significant others [20, 66, 76]. Using a ‘participant-centred approach’ during these multiple contacts, such as personalised telephone calls rather than generic reminder letters enhances retention [18, 20, 73, 123]. In a small pilot study with 48 methamphetamine injectors, Maher et al. [65] found mobile phone text messaging was a successful method of reaching participants (73% contacted) throughout the study. Participants reported that the text messaging was acceptable and there were no differences in the demographics between those who were retained and those who dropped out. Meyers et al. [66] provide a detailed analysis of the tracking strategies they used to obtain extremely high retention rates with substance-abusing youth. The authors described enhanced tracking efforts such as obtaining various contact details during their substance abuse treatment and post treatment, use of a locator form which recorded the contact details of family members, agencies they use and community locations they frequent. Small participant incentives were offered for follow-up (USD5) and approximately 40% of participants required six or more contacts in order to achieve follow-up interview completion. Most (60%) follow-up interviews were conducted in community settings preferred by participants such as fast food outlets. The strategies achieved 94% retention at one month follow-up and 92% retention at six month follow-up. The authors estimated that the additional tracking and retention strategies cost approximately USD85 per participant per follow-up wave.
Altruism and benefits of research
Some studies reported appealing to participants’ altruism in terms of the benefits the research might generate to people like themselves [30, 40, 71, 72]. This was often included in participant letters or reinforced during contact with interviewers.
Building relationships and trust
Flexibility in scheduling follow-up appointments, responding to messages positively, providing a caring environment, being courteous and frequent contact were all strategies designed to build relationships with participants and reduce attrition [16, 30, 35, 66, 70–72, 74, 76, 158]. Flexibility in scheduling appointments addresses practical barriers as well such as difficulties with child care or time from work. Brown et al. [72] reported that participants were more likely to remain in clinical trial research if their doctor helped them feel comfortable. Similarly McMillan et al. [96] credit high retention rates (78% at 6 weeks) of low-income pregnant women on the participants’ midwives involvement.
Establishing a toll-free number where participants can contact researchers free of cost has also been reported [39, 54]. Building a project community Signorello et al. [54] used an annual newsletter for longitudinal research participants providing updates on the research. Some studies reported appealing to participants’ altruism in terms of the benefits the research might generate to people like themselves, as well as for them [40, 71, 72]. This was often included in participant letters or reinforced during contact with interviewers.
Comparison studies
Two randomised trials of strategies to minimise attrition were found [23, 24]. Festinger et al. [23] randomly assigned participants who were receiving treatment for substance abuse to receive (USD) $70, $100, $130 or $160 incentives in either cash or a gift card for returning for a 6-month follow-up. The trial found that larger cash amounts resulted in the highest follow-up rates and fewer additional tracking calls [24].
Multiple strategies
It is worth noting that all studies used multiple strategies to maximise retention, commonly involving at least a comprehensive tracking strategy as well as incentives or gifts (Table 8).
Discussion
This is the first literature review to consider all of the research process where representativeness of socially disadvantaged groups may be compromised; sampling, recruitment, data collection, intervention delivery and retention. While previous reviews exist, most are now dated and focussed on one point in the research process, mainly the recruitment or retention of socially disadvantaged groups, and do not consider issues of engagement such as compliance with measurement and intervention. Previous reviews are also limited in scope focusing either on one target group of interest, one type of research study design or one type of intervention. The current review is greater in scope and comprehensiveness providing the reader with a complete picture of strategies for improving socially disadvantaged group participation in research for any given group, within any given research study type and using any given type of intervention strategy. Based on the outcomes of the 116 primary studies included in the current review and 31 literature reviews, a considerable number of barriers to the inclusion of socially disadvantaged groups in health and medical research were identified.
Strategies with good evidence of effectiveness were rare with only nine RCTs identified amongst the 116 studies. Based on this higher level evidence, there was no clear dominant strategy. The trials found mixed results for the effectiveness of incentives to enhance recruitment of socially disadvantaged groups into research [42, 43]. Similarly, variations in designs of recruitment letters for socially disadvantaged groups (e.g., hand written or printed [106]; generic or culturally sensitive [43]; and advance letter or no advance letter [36], found mixed results. One trial comparing data collection methods found that telephone interviews resulted in more complete data than mailed health surveys [90]. Two RCTs of strategies for improving intervention fidelity suggest that framing health information in a positive way emphasising progress achieved in the health area [37] and that group-based health counselling achieved greater outcomes than one-on-one counselling for hard-to-reach rural participants [88]. Finally, two trials of cash incentives found that larger cash amounts resulted in higher retention rates [23, 24].
A large number of strategies supported by fair and poor research evidence are outlined. One dominant theme was community engagement. Involving community groups and organisations in study design, sampling, recruitment, data collection, and intervention delivery was reported as essential by most studies improving recruitment, participation and retention.
Acknowledging the significance of community involvement in research with disadvantaged groups, the National Health and Medical Research Council (NHMRC) [159] in Australia requires that health research with Aboriginal participants includes substantial and formal community involvement at all levels [113, 114, 118]. An added benefit of community engagement in health research is the enhanced likelihood of facilitating the translation of research outcomes to policy and practice.
Quality of the evidence base
Synthesising the literature was made complex by the amount of methodological variation between included studies and the decision to combine a review of both barriers and strategies. Differences in study design and procedures, the nature of the health issue under investigation, settings and the target population groups limited comparisons across studies. A crude ‘levels of evidence’ hierarchy was used (as shown in Table 1), and it must be acknowledged that the usefulness of this approach in examining the research quality is limited. For example, while the RCT presents high level evidence of strategy effectiveness and qualitative research is considered low level evidence of strategy effectiveness, qualitative research, if conducted well, is appropriate research design for gaining an understanding of the barriers to research participation. However, when considering the studies included in the review which examined intervention effectiveness there were too few RCTs to pool data. As a result, this review does not provide conclusive evidence regarding the effectiveness of the discussed strategies. Some papers report case-studies or other descriptions of research which were not hypothesis-driven testing of strategies and not high level evidence of effectiveness. We included these studies as it is not possible to conduct RCTs of some multi-factorial strategies. Instead, these case studies provide valuable insights for researchers endeavouring to include socially disadvantaged groups, so long as the limitation in the quality of evidence is acknowledged. The strength of evidence for many of the strategies reported in this review is largely unknown and further robust experimental trials of those strategies which can be tested are required.
Implications for research practice
It is imperative that all types of health and medical research employ strategies to increase the representation of socially disadvantaged groups. Strategies will need to be designed and tailored according to different study types and research questions. In some cases, it is more appropriate to target the disadvantaged group of interest in the research. In other examples, like clinical trials of new medicines, it is important that various groups are represented in the overall study sample. It is clear from this review that the barriers to research with socially disadvantaged group are numerous and no one single solution to addressing the barriers exists. To address multiple barriers and challenges, a comprehensive, coordinated, multipronged approach involving many strategies across all stages of research needs to be adopted. This has significant time, cost and data ownership implications. Firstly, a long-term view to conducting research with socially disadvantaged groups is necessary. Developing relationships with communities and community groups, including their involvement in the development of procedures and study resources, and extensive formative research and pilot testing, require a considerable amount of time. Providing collaborating communities and groups with adequate feedback following the conclusion of the study, and in some cases, interventions or treatments, further extends project timelines. These activities ought to be reflected in project timelines and acknowledged by academic institutions employing researchers who conduct studies with socially disadvantaged groups. This long-term view should also be extended to publication of research with socially disadvantaged groups.
Secondly, many of the strategies are resource intensive and entail costs which may be additional to the usual costs of managing a research project. The addition of translation services or bilingual research staff, flexibility regarding data collection locations or times, gifts and incentives, culturally tailored resources and materials, extensive participant tracking procedures and additional staff training require consideration of costs. Some strategies such as including all family members in a study to avoid random selection [113], or offering those randomised to control conditions other types of interventions or treatments [148], may increase study costs significantly. These costs can be justified in funding applications and need to be recognised by funding organisations. Finally, in some cases of community collaborative research, data ownership may need to be negotiated and joint authorship with non-academic collaborators included. Academic researchers need to be supported by their academic institutions and scientific journals for publishing collaborative research which may take longer to produce due to authorship negotiations.
The establishment of research centres or research collaborations dedicated to high quality health and medical research with socially disadvantaged groups is one model for addressing many of the issues raised in this review in a coordinated manner [160, 161]. Numerous benefits would flow-on from these research centres including pooling of funding and resourcing, drawing on multidisciplinary expertise, promoting a high-level research culture in the field, expansion in the development of partnership links and networks with community organisations and groups which would increase access and recruitment, and training and building the capacity of future leaders in health research with socially disadvantaged groups.
Furthermore, a research collaborative could initiate and maintain a research participant registry for improved access to participants from numerous socially disadvantaged groups [25].
This review found a number of strategies and methodologies that may have a negative impact on health inequities. For example, the use of incentives, which have considerable evidence for improving recruitment and retention in the general literature [162], have negative connotations in some cultures [80]. Also, some commentators have raised ethical concerns regarding the use of incentives with research participants who may be financially compromised [163]. Festinger et al. [23, 24] however, provided compelling evidence in their trials of cash incentives with people in substance abuse treatment; participants in these trials did not use the cash incentive to purchase illicit substances and did not perceive the incentives as coercive highlighting the need to test strategies with target groups for both positive and negative effects. Similarly, the way language is used and information framed [37] can have a negative impact on the reactions of some groups to health research. It is difficult to generalise results from one disadvantaged group to another as different cultural factors may be influencing reactions and outcomes.
Implications for health policy and services
Knowledge gained from health research, particularly health services research, is used to inform the development or improvement of health policy and health care. If a diversity of the population of health service users is not included, service delivery is likely to be inequitable.
Doherty et al. [164] have identified three main hard-to-reach groups within service involvement: 1) the under-represented; 2) the invisible/overlooked; and 3) the service resistant. Reports suggest that the same types of barriers that result in underrepresentation in health research also lead to under-representation in health service use – distrust of authorities, health literacy and communication difficulties and pragmatic financial, transport or employment related barriers [165]. An improvement in the representation of vulnerable groups in health research, particularly health services research, is likely to yield benefits through more equitable service delivery and engagement.
Tudor-Hart first showed that the individual requiring the greatest effort to attend a health service is also the one with the greatest need [166]. While involving socially disadvantaged groups in activities may be resource-intensive and challenging, it has been argued that where sufficient funds allow the development of more creative approaches, it should be possible to engage with all members of society [167]. The strategies described in this review as potentially effective at increasing involvement in health research, may be generalizable to improving access and use of health services.
Study strengths and limitations
The main limitation of the study is the extent of methodological variation in the studies included in the review. The extent of heterogeneity between studies prohibited combining the results statistically in a meta-analysis. The inclusion of low level evidence also prohibits conclusive comments regarding the effectiveness of many of the strategies discussed in this review. However, this could also be considered a study strength as well; few RCTs were identified to provide a complete assessment of the effectiveness of strategies and inclusion of descriptive studies and case-studies provides valuable insights into ways to increase research with socially disadvantaged groups worth further testing. Finally, the majority of studies included in the review were conducted in developed countries and may not be generalizable to other countries.
Conclusion
Representativeness can be threatened at various stages of the research process, and researchers must remain mindful of whom they may be excluding in the design and implementation of research studies, and employ strategies to avoid this happening.
Researchers are in a powerful position to influence inequities in health outcomes. Generation of research findings that are representative of all social groups will allow development of an evidence base that can be used by service providers and policy makers to deliver programs and policies that reduce health inequities. Research funding agencies also play a role in ensuring that they fund research that demonstrates consideration of representativeness in its design, budget and timelines given this type of research is likely to be more complex, costly and time-consuming. However as the World Health Organisation’s Commission on the Social Determinants of Health notes the increased investment is highly worthwhile as addressing health inequities will result in health, social and economic benefits for all of society [10].
References
National Institute of Health (NIH): Guidelines on the inclusion of women and minorities as subjects in clinical research. NIH Guide. 1994, 23: 2-3.
Heiat A, Gross CP, Krumholz HM: Representation of the Elderly, Women, and Minorities in Heart Failure Clinical Trials. Arch Intern Med. 2002, 162 (15): doi:10.1001/archinte.162.15.1682
Johnson MS: Generalizability of clinical trials using homogenous samples – a metaanalysis. J Assoc Acad Minor Phys. 1990, 1: 31-33.
Rogers WA: Evidence based medicine and justice: a framework for looking at the impact of EBM upon vulnerable or disadvantaged groups. J Med Ethics. 2004, 30: 141-145. 10.1136/jme.2003.007062. doi: 10.1136/jme.2003.007062
Albain KS, Unger JM, Crowley JJ, Coltman CA, Hershman DL: Racial disparities in cancer survival among randomised clinical trial patients of the Southwest Oncology Group. J Natl Cancer Inst. 2009, 101: 984-992. 10.1093/jnci/djp175.
Singh GK, Azuine RE, Siahpush M: Global Inequalities in Cervical Cancer Incidence and Mortality are Linked to Deprivation, Low Socioeconomic Status, and Human Development. Int J MCH AIDS. 2012, 1 (1): 17-30.
Sateren WB, Trimble EL, Abrams J, Brawley O, Breen N, Ford L, McCabe M, Kaplan R, Smith M, Ungerlieder R, Christian MC: How sociodemographics, presence of oncology specialists, and hospital cancer programs affect accrual to cancer treatment trials. J Clin Oncol. 2002, 20: 2109-2117. 10.1200/JCO.2002.08.056.
Sydor A: Conducting research into hidden or hard-to-reach populations. Nurse Res. 2013, 20 (3): 33-37. 10.7748/nr2013.01.20.3.33.c9495.
The Collection and Interpretation of Data from Hidden Populations. Edited by: Lambert EY, Wiebel WW. 1990, Washington, DC: United States National Institute on Drug Abuse, from http://www.drugabuse.gov/pdf/monographs/download98.html
Marmot M, Friel S, Bell R, Houweling TAJ, Taylor S: Closing the gap in a generation: health equity through action on the social determinants of health. Lancet. 2008, 372: 1661-1669. 10.1016/S0140-6736(08)61690-6.
Guyatt GH, Sackett DL, Sinclair JC, Haywood R, Cook DJ, Cook RJ: Evidence-Based Medicine Working Group. Users’ guide to the medical literature IX. A method for grading health care recommendations. JAMA. 1995, 274: 1800-1804. 10.1001/jama.1995.03530220066035.
NHMRC: Guidelines for the Development and Implementation of Clinical Guidelines. 1995, Canberra: Australian Government Publishing Service
Evans D: Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs. 2003, 12: 77-84. 10.1046/j.1365-2702.2003.00662.x.
Hussain-Gambles M, Leese B, Atkin K, Brown J, Mason S, Tovey P: Involving South Asian patients in clinical trials. Health Technol Assess. 2004, 8 (42): 1-109.
Hussain-Gambles M, Atkin K, Leese B: South Asian participation in clinical trials: the views of lay people and health professionals. Health Policy. 2006, 77: 149-165. 10.1016/j.healthpol.2005.07.022.
Ashing-Giwa K, Rosales M: Recruitment and retention strategies of African American and Latina American breast cancer survivors in a longitudinal psychooncology study. Oncol Nurs Forum. 2012, 39 (5): e434-e442. 10.1188/12.ONF.E434-E442.
Odierna DH, Schmidt LA: The effects of failing to include hard-to-reach respondents in longitudinal surveys. Am J Public Health. 2009, 99 (8): 1515-1520. 10.2105/AJPH.2007.111138.
Escobar-Chaves SL, Tortolero SR, Masse LC, Watson KB, Fulton JE: Recruiting and retaining minority women: findings from the Women On The Move study. Ethn Dis. 2002, 12: 242-251.
Katz KS, Blake SM, Milligan RA, Sharps PW, White DB, Rodan MF, Rossi M, Murray KB: The design, implementation and acceptability of an integrated intervention to address multiple behavioural and psychosocial risk factors among pregnant African American women. BMC Pregnancy Childbirth. 2008, 8: 22-10.1186/1471-2393-8-22.
Rothschild SK, Martin MA, Swider SM, Lynas CT, Avery EF, Janssen I, Powell LH: The Mexican-American Trial of Community Health Workers (MATCH): design and baseline characteristics of a randomized controlled trial testing a culturally tailored community diabetes self-management intervention. Contemp Clin Trials. 2012, 33: 369-377. 10.1016/j.cct.2011.10.013.
Martin MA, Swider SM, Olinger T, Avery E, Lynas CMT, Carlson K, Rothschild SK: Recruitment of Mexican American adults for an intensive diabetes intervention trial. Ethn Dis. 2011, 21 (1): 7-12.
El-Khorazaty MN, Johnson AA, Kiely M, El-Mohandes AAE, Subramanian S, Laryea HA, Murray KB, Thornberry JS, Joseph JG: Recruitment and retention of low-income minority women in a behavioural intervention to reduce smoking, depression, and intimate partner violence during pregnancy. BMC Public Health. 2007, 7: 233-10.1186/1471-2458-7-233.
Festinger DS, Marlowe DB, Croft JR, Dugosh KL, Mastro NK, Lee PA, DeMatteo DS, Patapis NS: Do research payments precipitate drug use or coerce participation?. Drug Alcohol Depend. 2005, 78: 275-281. 10.1016/j.drugalcdep.2004.11.011.
Festinger DS, Marlowe DB, Dugosh KL, Croft JR, Arabia PL: Higher magnitude cash payments improve research follow-up rates without increasing drug use or perceived coercion. Drug Alcohol Depend. 2008, 96: 128-135. 10.1016/j.drugalcdep.2008.02.007.
Green MA, Kim MM, Barber S, Odulana AA, Godley PA, Howard DL, Corbie-Smith GM: Connecting communities to health research: development of the Project CONNECT minority research registry. Contemp Clin Trials. 2013, 35: 1-7.
Sutherland MA, Fantasia HC: Successful research recruitment strategies in a study focused on abused rural women at risk for sexually transmitted infections. J. Midwifery Womens Health. 2012, 57: 381-385. 10.1111/j.1542-2011.2011.00134.x.
Goldade K, Whembolua G, Thomas J, Eischen S, Guo H, Connett J, Jarlais DD, Resnicow K, Gelberg L, Owen G, Grant J, Ahluwalia JS, Okuyemi KS: Designing a smoking cessation intervention for the unique needs of homeless persons: a community-based randomised clinical trial. Clin Trials. 2011, 8: 744-754. 10.1177/1740774511423947.
Germino BB, Mishel MH, Alexander R, Jenerette C, Blyler D, Baker C, Vines AI, Green M, Long DG: Engaging African American breast cancer survivors in an intervention trial: culture, responsiveness and community. J Cancer Surviv. 2011, 5: 82-91. 10.1007/s11764-010-0150-x.
Fu SS, Rhodes KL, Robert C, Widome R, Forster JL, Joseph AM: Designing and evaluating culturally specific smoking cessation interventions for American Indian communities. Nicotine Tob Res. 2013, Advance Access, doi:10.1093/ntc/ntt111
Webb DA, Coyne JC, Goldenberg RL, Hogan VK, Elo IT, Bloch JR, Mathew L, Bennett IM, Dennis EF, Culhane JF: Recruitment and retention of women in a large randomized control trial to reduce repeat preterm births: the Philadelphia Collaborative Preterm Prevention Project. BMC Med Res Methodol. 2010, 10: 88-10.1186/1471-2288-10-88.
Ammerman A, Washington C, Jackson B, Weathers B, Campbell M, Davis G, Garson-Angert D, Paige J, Parks-Bani C, Joyner M, Keyserling T, Switzer B: The Praise! Project: a church-based nutrition intervention designed for cultural appropriateness, sustainability and diffusion. Health Promot Pract. 2002, 3: 286-301.
McCaskill-Stevens W, Pinto H, Comis R, Morgan R, Plomer K, Schoentgen S: Recruiting minority cancer patients into cancer clinical trials: a pilot project involving the Eastern cooperative oncology group and the National Medical Association. J Clin Oncol. 1999, 17 (3): 1029-1039.
Daniulaityte R, Falck R, Li L, Nahhas RW, Carlson RG: Respondent-driven sampling to recruit young adult non-medical users of pharmaceutical opioids: Problem and solutions. Drug Alcohol Depend. 2012, 121: 23-29. 10.1016/j.drugalcdep.2011.08.005.
Peterson JA, Reisinger HS, Schwartz RP, Mitchell SG, Kelly SM, Brown BS, Agar MH: Targeted sampling in drug abuse research: a review and case study. Field Meth. 2008, 20: 155-170. 10.1177/1525822X08314988.
Morse EV, Simon PM, Besch L, Walker J: Issues of recruitment, retention and compliance in community-based clinical trials with traditionally underserved populations. Appl Nurs Res. 1995, 8 (1): 8-14. 10.1016/S0897-1897(95)80240-1.
Napoles-Springer AM, Fongwa MN, Stewart AL, Gildengorin G, Perez-Stable EJ: The effectiveness of an advance notice letter on the recruitment of African Americans and Whites for a mailed patient satisfaction survey. J Aging Health. 2004, 16: S124-S136. 10.1177/0898264304269724.
Nicholson RA, Kreuter MW, Lapka C, Wellborn R, Clark EM, Sanders-Thompson V, Jacobsen HM, Casey C: Unintended effects of emphasizing disparities in cancer communication to African-Americans. Canc Epidemiol Biomarkers Prev. 2008, 17: 2946-2953. 10.1158/1055-9965.EPI-08-0101.
Stueve A, O’Donnell LN, Duran R, Doval AS, Blome J: Time-space sampling in minority communities: Results with young Latino men who have sex with men. J Public Health. 2001, 91 (6): 922-926.
McCuller WJ, Sussman S, Holiday K, Craig S, Dent CW: Tracking procedures for locating high-risk youth. Eval Health Prof. 2002, 25: 345-362. 10.1177/0163278702025003006.
Hough RL, Tarke H, Renker V, Shields P, Glatstein J: Recruitment and retention of homeless mentally ill participants in research. J Consult Clin Psychol. 1996, 64 (5): 881-891.
Harris KJ, Ahluwalia JS, Catley D, Okuyemi KS, Mayo MS, Resnicow K: Successful recruitment of minorities into clinical trials: the Kick It at Swope project. Nicotine Tob Res. 2003, 5 (4): 575-584. 10.1080/1462220031000118540.
Maxwell AE, Bastani R, Glenn BA, Mojica CM, Chang C: An experimental test of the effect of incentives on recruitment of ethnically diverse colorectal cancer cases and their first-degree relatives into a research study. Canc Epidemiol Biomarkers Prev. 2009, 18 (10): 2620-2625. 10.1158/1055-9965.EPI-09-0299.
Satia JA, Galanko JA, Rimer BK: Methods and strategies to recruit African Americans into cancer prevention surveillance studies. Canc Epidemiol Biomarkers Prev. 2005, 14: 718-721. 10.1158/1055-9965.EPI-04-0132.
Derose KP, Hawes-Dawson J, Fox SA, Maldonado N, Tatum A, Kingyon R: Dealing with diversity: recruiting churches and women for a randomized trial of mammography promotion. Health Educ Behav. 2000, 27: 632-648. 10.1177/109019810002700508.
Sudore RL, Landefeld S, Williams BA, Barnes DE, Lindquist K, Schillinger D: Use of a modified informed consent process among vulnerable patients. A descriptive study. J Gen Intern Med. 2006, 21: 867-873. 10.1111/j.1525-1497.2006.00535.x.
Hendrickson SG: Video recruitment of non-English speaking participants. West J Nurs Res. 2007, 29: 232-242. 10.1177/0193945906292558.
Hatchett BF, Holmes K, Duran DA, Davis C: African Americans and research participation: The recruitment process. J Black Stud. 2000, 30: 664-674. 10.1177/002193470003000502.
Hahn EA, Cella D: Health outcomes assessment in vulnerable populations: measurement challenges and recommendations. Arch Phys Med Rehabil. 2003, 84 (S2): 35-42.
Shebl F, Poppell CF, Zhan M, Dwyer DM, Hopkins AB, Groves C, Reed F, Devadason C, Steinberger EK: Measuring health behaviours and landline telephones: potential coverage bias in a low-income rural population. Public Health Rep. 2009, 124 (4): 495-502.
Adams-Campbell LL, Ahaghotu C, Gaskins M, Dawkins FW, Smoot D, Polk OD, Gooding R, Dewitty RL: Enrolment of African Americans onto clinical treatment trials: study design barriers. J Clin Oncol. 2004, 22: 730-734. 10.1200/JCO.2004.03.160.
Mathy RM, Schillace M, Coleman SM, Berquist BE: Methodological rigor with internet samples: new ways to reach underrepresented populations. Cyberpsychol Behav. 2002, 5 (3): 253-266. 10.1089/109493102760147259.
Chang BL, Bakken S, Brown S, Houston TK, Kreps GL, Kukafka R, Safran C, Stavri Z: Bridging the digital divide: reaching vulnerable populations. J Am Med Inform Assoc. 2004, 11: 448-457. 10.1197/jamia.M1535.
Martsolf DS, Courey TJ, Chapman TR, Drauker CB, Mims BL: Adaptive sampling: recruiting a diverse community sample of survivors of sexual violence. J Community Health Nurs. 2006, 23 (3): 169-182. 10.1207/s15327655jchn2303_4.
Signorello LB, Hargreaves MK, Steinwandel MD, Zheng W, Cai Q, Schlundt DG, Buchowski MS, Arnold CW, McLaughlin JK, Blot WJ: Southern community cohort study: establishing a cohort to investigate health disparities. J Natl Med Assoc. 2005, 97 (7): 972-978.
Harper GW, Carver LJ: “Out-of-the-Mainstream” Youth Partners in Collaborative research: exploring the benefits and challenges. Health Educ Behav. 1999, 26: 250-265. 10.1177/109019819902600208.
Han H, Kang J, Kim K, Ryu J, Kim M: Barriers to and strategies for recruiting Korean Americans for community-partnered health promotion research. J Immigr Minor Health. 2007, 9: 137-146. 10.1007/s10903-006-9022-x.
Sadler GR, Lee H, Lim RS, Fullerton J: Recruiting hard-to-reach United States population sub-groups via adaptations of snowball sampling strategy. Nurs Health Sci. 2010, 12 (3): 369-374. 10.1111/j.1442-2018.2010.00541.x.
Lacey L, Tukes S, Manfredi C, Warnecke RB: Use of lay health educators for smoking cessation in a hard-to-reach urban community. J Community Health. 1991, 16 (5): 269-281. 10.1007/BF01320335.
Alemagno SA, Cochran D, Feucht TE, Stephens RC, Butts JM, Wolfe SA: Assessing substance abuse treatment needs among the homeless: a telephone-based interactive voice response system. Am J Public Health. 1996, 86: 1626-1628. 10.2105/AJPH.86.11.1626.
Shedlin MG, Decena CU, Mangadu T, Martinez A: Research participant recruitment in Hispanic communities: lessons learned. J Immigr Minor Health. 2011, 13: 352-360. 10.1007/s10903-009-9292-1.
Graham AL, Lopez-Class M, Mueller NT, Mota G, Mandelblatt J: Efficiency and costeffectiveness of recruitment methods for male Latino smokers. Health Educ Behav. 2011, 38 (3): 293-300. 10.1177/1090198110372879.
Keyzer JF, Melnikow J, Kuppermann M, Birch S, Kuenneth C, Nuovo J, Azari R, Oto-Kent D, Rooney M: Recruitment strategies for minority participation: challenges and cost lessons from the POWER interview. Ethn Dis. 2005, 15: 395-406.
McGraw SA, McKinlay JB, Crawford SL, Costa LA, Cohen DL: Health survey methods with minority populations: some lessons from recent experience. Ethn Dis. 1992, 2 (3): 273-287.
Newman PA, Duan N, Roberts KJ, Seiden D, Rudy ET, Swendeman D, Popova S: HIV vaccine trial participation among ethnic minority communities; Barriers, Motivators and Implications for recruitment. J Acquir Immune Defic Syndr. 2006, 41: 210-217. 10.1097/01.qai.0000179454.93443.60.
Maher JE, Pranian K, Drach L, Rumptz M, Casciato C, Guernsey J: Using text messaging to contact difficult-to-reach study participants. Am J Public Health. 2010, 100 (6): 969-970. 10.2105/AJPH.2009.188391.
Meyers K, Webb A, Frantz J, Randall M: What does it take to retain substanceabusing adolescents in research protocols? Delineation of effort required, strategies undertaken, costs incurred, and 6-month post-treatment differences by retention difficulty. Drug Alcohol Depend. 2003, 69: 73-85. 10.1016/S0376-8716(02)00252-1.
Alvarez RA, Vasquez E, Mayorga CC, Feaster DJ, Mitrani VB: Increasing minority research participation through community organisation outreach. West J Nurs Res. 2006, 28 (5): 541-563. 10.1177/0193945906287215.
Atkinson NL, Billing AS, Desmond SM, Gold RS, Tournas-Hardt A: Assessment of the nutrition and physical activity education needs of low-income, rural mothers: can technology play a role?. J Community Health. 2007, 32 (4): 245-267. 10.1007/s10900-007-9047-7.
Atkinson NL, Saperstein SL, Desmond SM, Gold RS, Billing AS, Tian J: Rural ehealth nutrition education for limited-income families: an iterative and user-centred design approach. J Med Internet Res. 2009, 11 (2): e21-10.2196/jmir.1148.
Burns D, Soward ACM, Skelly AH, Leeman J, Carlson J: Effective recruitment and retention strategies for older members of rural minorities. Diabetes Educ. 2008, 34: 1045-1051. 10.1177/0145721708325764.
Loftin WA, Barnett SK, Bunn PS, Sullivan P: Recruitment and retention of rural African Americans in Diabetes Research: Lessons Learned. Diabetes Educ. 2005, 31: 251-259. 10.1177/0145721705275517.
Brown DR, Fouad MN, Basen-Engquist K, Tortolero-Luna G: Recruitment and retention of minority women in cancer screening, prevention and treatment trials. Ann Epidemiol. 2000, 10: S13-S21. 10.1016/S1047-2797(00)00197-6.
Woods MN, Harris KJ, Mayo MS, Catley D, Scheibmeir M, Ahluwalia JS: Participation of African Americans in a smoking cessation trial: a quantitative and qualitative study. JAMA. 2002, 94 (7): 609-618.
Leach CR, Schoenberg NE, Hatcher J: Factors associated with participation in cancer prevention and control studies among rural Appalachian women. Fam Community Health. 2011, 34 (2): 119-125.
Blumenthal DS, Sung J, Coates R, Williams J, Liff J: Recruitment and retention of subjects for a longitudinal cancer prevention study in an inner-city black community. Health Serv Res. 1995, 30 (1): 197-205.
Brown-Peterside P, Rivera E, Lucy D, Slaughter I, Ren L, Chiasson MA, Koblin BA: Retaining hard-to-reach women in HIV prevention and vaccine trials: project ACHIEVE. Am J Public Health. 2001, 91 (9): 1377-1379. 10.2105/AJPH.91.9.1377.
Chen AM, Wismer BA, Lew R, Hang SH, Min K, Moscowitz JM, Tager IB: “Health is Strength”: a research collaboration involving Korean Americans in Alameda County. Am J Prev Med. 1997, 13 (S6): 93-100.
Israel BA, Krieger J, Vlahov D, Ciske S, Foley M, Fortin P, Guzman RJ, Lichtenstein R, McGranaghan R, Palermo A, Tang G: Challenges and facilitating factors in sustaining community-based participatory research partnerships: lessons learned from the Detroit New York City and Seattle Urban Research Centres. Bull New York Acad Med. 2006, 83 (6): 1022-1040.
Robinson WT, Risser JMH, McGoy S, Becker AB, Rehman H, Jefferson , Griffin V, Wolverton M, Tortu S: Recruiting injection drug users: a three site comparison of results and experiences with respondent-driven and targeted sampling procedures. J Urban Health Bull New York Acad Med. 2006, 83 (7): i29-i38.
Lindenberg CS, Solorzano RM, Vilaro FM, Westbrook LO: Challenges and strategies for conducting intervention research with culturally diverse populations. J Transcult Nurs. 2001, 12: 132-139. 10.1177/104365960101200207.
Faseru B, Cox LS, Bronars CA, Opole I, Reed GA, Mayo MS, Ahluwalia JS, Okuyemi KS: Design, recruitment, and retention of African-American smokers in a pharmacokinetic study. BMC Med Res Methodol. 2010, 10: 6-10.1186/1471-2288-10-6.
Hughes JJ: Paying injection drug users to educate and recruit their peers: Why participant-driven interventions are an ethical public health model. Qual Manag Health Care. 1999, 7 (4): 4-12. 10.1097/00019514-199907040-00004.
Grov C, Bux D, Parsons JT, Morgenstern J: Recruiting hard-to-reach drug-using men who have sex with men into an intervention study: lessons learned and implications for applied research. Subst Use Misuse. 2009, 44: 1855-1871. 10.3109/10826080802501570.
Goel S, Salganik MJ: Assessing respondent-driven sampling. Proc Natl Acad Sci U S A. 2010, 107 (15): 6743-6747. 10.1073/pnas.1000261107.
Eakin EG, Bull SS, Riley K, Reeves M, Gutierrez S, McLaughlin P: Recruitment and retention of Latinos in a primary care-based physical activity and diet trial: the Resources for Health study. Health Educ Res. 2006, 22 (3): 361-371. 10.1093/her/cyl095.
Cortes DE, Drainoni M-L, Henault LE, Paasche-Orlow MK: How to achieve informed consent for research from Spanish-speaking individuals with low literacy: a qualitative report. J Health Commun. 2010, 15 (S2): 172-182.
Clayman ML, Mangello JA, Viswanath K, Hesse BW, Arora NK: Providing health messages to Hispanics/Latinos: understanding the importance of language, trust in health information sources, and media use. J Health Commun. 2010, 15 (S3): 252-263.
Befort CA, Donnelly JE, Sullivan DK, Ellerbeck EF, Perri MG: Group versus individual phone-based obesity treatment for rural women. Eat Behav. 2010, 11: 11-17. 10.1016/j.eatbeh.2009.08.002.
Robinson JM, Trochim WMK: An examination of community members’, researchers’ and health professionals’ perceptions of barriers to minority participation in medical research: an application of concept mapping. Ethn Health. 2007, 12 (5): 521-539. 10.1080/13557850701616987.
Ngo-Metzer Q, Kaplan S, Sorkin DH, Claridge BR, Phillips RS: Surveying minorities with limited-English proficiency. Does data collection method affect data quality among Asian Americans?. Med Care. 2004, 42 (9): 893-900. 10.1097/01.mlr.0000135819.15178.bc.
Kalsbeek WD: Sampling minority groups in health surveys. Stat Med. 2003, 22: 1527-1549. 10.1002/sim.1510.
Rooney LK, Bhopal R, Halani L, Levy ML, Partridge MR, Netuveli G, Car J, Griffiths C, Atkinson J, Lindsay G, Sheikh A: Promoting recruitment of minority ethnic groups into research: qualitative study exploring the views of South Asian people with asthma. J Public Health. 2011, 33 (4): 604-615. 10.1093/pubmed/fdq100.
Rugkasa J, Canvin K: Researching mental health in minority ethnic communities: reflections on recruitment. Qual Health Res. 2011, 21: 132-10.1177/1049732310379115.
Stirland L, Halani L, Raj B, Netuveli G, Partridge M, Car J, Griffiths C, Levy M, Sheikh A: Recruitment of South Asians into asthma research: qualitative study of UK and US researchers. Prim Care Respir J. 2011, 20 (3): 282-290. 10.4104/pcrj.2011.00032.
Bloor M: Population estimation without censuses or surveys: a discussion of markrecapture methods illustrated by results from three studies. Sociology. 2005, 39: 121-138.
McMillan B, Green JM, Woolridge MW, Dyson L, Renfrew MJ, Clarke GP: Studying the infant feeding intentions of pregnant women experiencing material deprivation: Methodology of the Looking at Infant Feeding Today (LIFT) study. Soc Sci Med. 2009, 68: 845-849. 10.1016/j.socscimed.2008.12.032.
Dowrick C, Gask L, Edwards S, Aseem S, Bower P, Burroughs H, Catlin A, Chew-Graham C, Clarke P, Gabbay M, Gowers S, Hibbert D, Kovandzic M, Lamb J, Lovell K, Rogers A, Lloyd-Williams M, Waheed W: Researching the mental health needs of hard-to-reach groups: Managing multiple sources of data. BMC Health Serv Res. 2009, 9: 226-10.1186/1472-6963-9-226.
Elliot E, Watson AJ, Harries U: Harnessing expertise: involving peer interviewers in qualitative research with hard-to-reach populations. Health Expect. 2002, 5: 172-178. 10.1046/j.1369-6513.2002.00158.x.
Kiezebrink K, Crombie IK, Irvine L, Swanson V, Power K, Wrieden WL, Slane PW: Strategies for achieving a high response rate in a home interview survey. BMC Med Res Methodol. 2009, 9: 46-10.1186/1471-2288-9-46.
Anderson M, Solarin I, Gerver S, Elam G, MacFarlane E, Fenton K, Easterbrook P: The LIVITY study: research challenges and strategies for engaging with black Caribbean community in a study of HIV infection. Int J Soc Res Methodol. 2009, 12 (3): 197-209. 10.1080/13645570701708584.
Sheikh A, Halani L, Bhopal R, Netuveli G, Partridge MR, Car J, Griffiths C, Levy M: Facilitating the recruitment of minority ethnic people into research: qualitative case study of South Asians and asthma. PLoS Med. 2009, 6 (10): e1000148-10.1371/journal.pmed.1000148.
Browne K: Snowball sampling: using social networks to research nonheterosexual women. Int J Soc Res Methodol. 2007, 8 (1): 47-60.
Hoppitt T, Shah S, Bradburn P, Gill P, Calvert M, Pall H, Stewart M, Fazil Q, Sackley C: Reaching the ‘hard to reach’: strategies to recruit black and minority ethnic service users with rare long-term neurologic conditions. Int J Soc Res Methodol. 2012, 15 (6): 485-495. 10.1080/17522439.2011.615161.
Hughes AO, Fenton S, Hine CE, Pilgrim S, Tibbs N: Strategies for sampling black and ethnic minority populations. J Publ Health Med. 1995, 17 (2): 187-192.
Ryan L, Kofman E, Aaron P: Insiders and outsiders: working with peer researchers in researching Muslim communities. Int J Soc Res Methodol. 2011, 14 (1): 49-60. 10.1080/13645579.2010.481835.
Choudhury Y, Hussain I, Parsons S, Rahman A, Eldridge S, Underwood M: Methodological challenges and approaches to improving response rates in population surveys in areas of extreme deprivation. Prim Health Care Res Dev. 2012, 13: 211-218. 10.1017/S1463423611000466.
Greenhalgh T, Collard A, Begum N: Sharing stories: complex intervention for diabetes education in minority ethnic groups who do not speak English. BMJ. 2005, 330: 1-6. 10.1136/bmj.330.7481.1.
Chiu LF, West RM: Health intervention in social context: Understanding social networks and neighbourhood. Soc Sci Med. 2007, 65: 1915-1927. 10.1016/j.socscimed.2007.05.035.
McLean CA, Campbell CM: Locating research informants in a multi-ethnic community: ethnic identities, social networks and recruitment methods. Ethn Health. 2003, 8 (1): 41-61. 10.1080/13557850303558.
Allison T, Ahmad T, Brammah T, Symmons D, Urwin M: Can findings from postal questionnaires be combined with interview results to improve the response rate among ethnic minority populations?. Ethn Health. 2003, 8 (1): 63-69. 10.1080/13557850303552.
Elam G, Fenton KA: Researching sensitive issues and ethnicity: lessons from sexual health. Ethn Health. 2003, 8 (1): 15-27. 10.1080/13557850303557.
Oakley A, Wiggins M, Turner H, Rajan L, Barker M: Including culturally diverse samples in health research: a case study of an urban trial of social support. Ethn Health. 2003, 8 (1): 29-39. 10.1080/13557850303554.
Fox S, Arnold A, Dunn R, Keeffe J, Taylor H: Sampling and recruitment methodology for a national eye health survey of Indigenous Australians. Aust New Zeal J Publ Health. 2010, 34: 554-562. 10.1111/j.1753-6405.2010.00635.x.
Couzos S, Lea T, Murray R, Culbong M: “We are not just participants – We are in charge”: the NACCHO Ear Trial and the process for Aboriginal communitycontrolled Health Research. Ethn Health. 2005, 10 (2): 91-111. 10.1080/13557850500071038.
Hing N, Breen H, Gordon A: Respecting cultural values: conducting a gambling survey in an Australian Indigenous community. Aust New Zeal J Publ Health. 2010, 34: 547-553. 10.1111/j.1753-6405.2010.00624.x.
Lennox N, Taylor M, Rey-Conde T: Beating the barriers: recruitment of people with intellectual disability to participate in research. J Intellect Disabil Res. 2005, 49 (4): 296-305. 10.1111/j.1365-2788.2005.00618.x.
Thomas D, Fitz JW, Johnston V, Townsend J, Kneebone W: Wholesale data surveillance of Australian Aboriginal tobacco consumption in the Northern Territory. Tob Control. 2011, 20: 291-295. 10.1136/tc.2010.042473.
Isaacs AN, Pepper H, Pyett P, Gruis HA, Waples-Crowe P, Oakley-Browne MA: What you do is important but how you do it is more important. Engaging Indigenous men in rural mental health services research. Qual Res J. 2011, 11 (1): 51-61. 10.3316/QRJ1101051.
Cleland V, Ball K: Recruiting hard-to-reach populations: lessons from a study of women living in socioeconomically disadvantaged areas of Victoria, Australia. Health Promot J Austr. 2010, 21 (3): 243-244.
Bryant J, Bonevski B, Paul C, Lecathelinais C: Assessing smoking status in disadvantaged populations: is computer administered self-report an accurate and acceptable measure?. BMC Med Res Methodol. 2011, 11: 153-10.1186/1471-2288-11-153.
Kelly J, Saggers S, Taylor K, Pearce G, Massey P, Bull J, Odo T, Thomas J, Billycan R, Judd J, Reilly S, Ahboo S: “Makes you proud to be black, eh?”: Reflections on meaningful Indigenous research participation. Int J Equity Health. 2012, 11: 40-10.1186/1475-9276-11-40.
Benoit C, Jansson M, Millar A, Philips R: Community-academic research on hardto-reach populations: benefits and challenges. Qual Health Res. 2005, 15: 263-282. 10.1177/1049732304267752.
Furimsky I, Cheung AH, Dewa CS, Zipursky RB: Strategies to enhance patient recruitment and retention in research involving patients with a first episode of mental illness. Contemp Clin Trials. 2008, 29: 862-866. 10.1016/j.cct.2008.07.005.
Greene S, Ahluwalia A, Watson J, Tucker R, Rourke SB, Koornstra J, Sobota M, Monette L, Byers S: Between scepticism and empowerment: the experiences of peer research assistants in HIV/AIDS, housing and homelessness community-based research. Int J Soc Res Methodol. 2009, 12 (4): 361-373. 10.1080/13645570802553780.
Wong ST, Bowell B, Housden L, Lavoi J: Strategies for moving towards equity in recruitment of rural and Aboriginal research participants. Rural Remote Health. 2013, 13: 2453-
Platt L, Wall M, Rhodes T, Judd A, Hickman M, Johnston LG, Renton A, Bobrova N, Sarang A: Methods to recruit hard-to-reach groups: comparing two chain referral sampling methods of recruiting injecting drug users across nine studies in Russia and Estonia. J Urban Health Bull New York Acad Med. 2006, 83 (7): i39-i53.
Dingoyan D, Schulz H, Mosko M: The willingness to participate in health research studies of individuals with Turkish migration backgrounds: barriers and resources. Eur Psychiatry. 2012, 27 (S2): S4-S9.
Mhurchu CN, Blakely T, Funaki-Tahifote M, McKerchar C, Wilton J, Chua S, Jiang Y: Inclusion of indigenous and ethnic minority populations in intervention trials: challenges and strategies in a New Zealand supermarket study. J Epidemiol Community. 2009, 63: 850-855. 10.1136/jech.2008.081109.
Abramovitz D, Volz EM, Strathdee SA, Patterson TL, Vera A, Frost SDW: Using respondent driven sampling in a hidden population at risk of HIV infection: Who do HIV-positive recruiters recruit?. Sex Transm Dis. 2009, 36 (12): 750-756. 10.1097/OLQ.0b013e3181b0f311.
Sheridan SL, Halpern DJ, Viera AJ, Berkman ND, Donahue KE, Crotty K: Interventions for individuals with low health literacy: a systematic review. J Health Commun Int Perspect. 2011, 16 (S3): 30-54.
Swanson GM, Ward AJ: Recruiting minorities into clinical trials: toward a participant-friendly system. J Natl Cancer Inst. 1995, 87 (23): 1747-1759. 10.1093/jnci/87.23.1747.
Ford JG, Howerton MW, Lai GY, Gary TL, Bolen S, Gibbons MC, Tilburt J, Baffi C, Tanpitukpongse TP, Wilson R, Powe NR, Bass EB: Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer. 2008, 112: 228-242. 10.1002/cncr.23157.
Hergenrather KC, Rhodes SD, Cowan CA, Bardhoshi G, Pula S: Photovoice as a community-based participatory research: a qualitative review. Am J Health Behav. 2009, 33 (6): 686-698.
Halcomb EJ, Gholizadeh L, DiGiacomo M, Philips J, Davidson P: Literature review: considerations in undertaking focs group research with culturally and linguistically diverse groups. J Clin Nurs. 2007, doi: 10.1111/j.1365-2702.23006.01760.x
Grove N, Brough M, Canuto C, Dobson A: Aboriginal and Torres Strait Islander health research and the conduct of longitudinal studies: issues for debate. Aust New Zeal J Publ Health. 2003, 27 (6): 637-641. 10.1111/j.1467-842X.2003.tb00612.x.
Booth S: Researching health and homelessness: methodological challenges for researchers working with a vulnerable, hard-to-reach, transient population. Aust J Prim Health Interchange. 1999, 5 (3): 76-81. 10.1071/PY99037.
Mathews J, Cramer EP: Using technology to enhance qualitative research with hidden populations. Qual Rep. 2008, 13 (2): 301-315.
Andresen EM, Diehr PH, Luke DA: Public Health Surveillance of low-frequency populations. Annu Rev Publ Health. 2004, 25: 25-52. 10.1146/annurev.publhealth.25.101802.123111.
Shavers-Hornaday VL, Lynch CF, Burmeister LF, Torner JC: Why are African Americans under-represented in medical research studies? Impediments to participation. Ethn Health. 1997, 2 (1–2): 31-45.
Flory J, Ezekiel E: Interventions to improve research participants’ understanding in informed consent for research: a systematic review. JAMA. 2004, 292 (13): 1593-1601. 10.1001/jama.292.13.1593.
UyBico SJ, Pavel S, Gross CP: Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007, 22: 852-863. 10.1007/s11606-007-0126-3.
Lai GY, Gary TL, Tilburt J, Bolen S, Baffi C, Wilson RF, Howerton MW, Gibbons MC, Tanpitukpongse TP, Powe NR, Bass EB, Ford JG: Effectiveness of strategies to recruit underrepresented populations into cancer clinical trials. Clin Trials. 2006, 3: 133-141. 10.1191/1740774506cn143oa.
Howerton MW, Gibbons MC, Baffi CR, Gary TL, Lai GY, Bolen S, Tilburt J, Tanpitukpongse TP, Wilson RF, Powe N, Bass EB, Ford JG: Provider roles in the recruitment of underrepresented populations to cancer clinical trials. Cancer. 2007, 109: 465-476. 10.1002/cncr.22436.
Giuliano AR, Mokuau N, Hughes C, Tortolero-Luna G, Risendal B, Ho RCS, Prewitt TE, McKaskill-Stevens WJ: Participation of minorities in cancer research: the influence of structural, cultural and linguistic factors. AEP. 2000, 10 (S8): S22-S34.
Miranda J, Azocar F, Organista KC, Munoz RE, Lieberman A: Recruiting and retaining low-income Latinos in psychotherapy research. J Consult Clin Psychol. 1996, 64 (5): 868-874.
Wendler D, Kington R, Madans J, Van Wye G, Christ-Schmidt H, Pratt LA, Brawley OW, Gross CP, Emanuel E: Are racial and ethnic minorities less willing to participate in health research?. Plos Med. 2006, 3 (2): e19-10.1371/journal.pmed.0030019. doi: 10.1371/journal.pmed.0030019
Dhalla S, Poole G: Motivators of enrolment in HIV vaccine trials: a review of HIV vaccine preparedness studies. AIDS Care. 2011, 23 (11): 1430-1447. 10.1080/09540121.2011.555750.
Yancey AK, Ortega AN, Kumanyika SK: Effective recruitment and retention of minority research participants. Annu Rev Publ Health. 2006, 27: 1-28. 10.1146/annurev.publhealth.27.021405.102113.
Kalton G: Methods for oversampling rare subpopulations in social surveys. Surv Meth. 2009, 35 (2): 125-141.
Atkinson R, Flint J: Accessing hidden and hard-to-reach populations: snowball research strategies. Soc Res Update. 2001, 33 (1): 1-4.
Malekinejad M, Johnston LG, Kendall C, Kerr LRFS, Rifkin MR, Rutherford GW: Using respondent-driven sampling methodology for HIV biological and behavioural surveillance in international settings: a systematic review. AIDS Behav. 2008, 12: S105-S130. 10.1007/s10461-008-9421-1.
Marpsat M, Razafindratsima N: Survey methods for hard-to-reach populations: introduction to the special issue. Methodol Innov Online. 2010, 5 (2): 3-16.
Aldana BU, Quintero MA: A comparison of three methods for sampling hard-toreach or hidden populations. Pensamiento Psicologico. 2008, 4 (10): 167-176.
Faugier J, Sargeant M: Sampling hard to reach populations. J Adv Nurs. 1997, 26: 790-797. 10.1046/j.1365-2648.1997.00371.x.
Johnson VA, Powell-Young YM, Torres ER, Spruill IJ: A systematic review of strategies that increase the recruitment and retention of African American Adults in genetic and genomic studies. ABNF J. 2011, 22 (4): 84-88.
Ndumele CD, Ableman G, Russell BE, Gurrola E, Hicks LS: Publication of recruitment methods in focus group research of minority populations with chronic disease: a systematic review. J Health Care Poor Underserved. 2011, 22 (1): 5-23.
Glazier RH, Bajcar J, Kennie NR, Willson K: A systematic review of interventions to improve diabetes care in socially disadvantaged populations. Diabetes Care. 2006, 29: 1675-1688. 10.2337/dc05-1942.
Wallace DC, Bartlett R: Recruitment and retention of African American and Hispanic girls and women in research. Public Health Nurs. 2012, 30 (2): 159-166.
National Health and Medical Research Council: Values and Ethics: Guidelines for Ethical Conduct in Aboriginal and Torres Strait Islander Health Research. 2003, Canberra: Commonwealth of Australia
Leeman J, Harrell JS, Funk SG: Building a research program focussed on vulnerable people. West J Nurs Res. 2002, 24: 103-111. 10.1177/01939450222045743.
Sanson-Fisher R, Brand M, Shakeshaft A, Haber P, Day C, Conigrave K, Mattick R, Lintzeris N, Teesson M: Forming a national multicentre collaboration to conduct clinical trials: increasing high-quality research in the drug and alcohol field. Drug Alcohol Rev. 2010, 29: 469-474. 10.1111/j.1465-3362.2009.00166.x.
Caldwell PHY, Hamilton S, Tan A, Craig JC: Strategies for increasing recruitment to randomised controlled trials: systematic review. PLoS Med. 2010, 7 (11): e1000368-10.1371/journal.pmed.1000368.
Popay J: Should disadvantaged people be paid to take care of their health? No. BMJ. 2008, 337: a594-10.1136/bmj.a594.
Doherty PA, Scott A, Kinder K: Delivering services to hard to reach families in on track areas: definition, consultation and needs assessment. Home Office Development and Practice Report. 15: http://www.homeoffice.gov.uk/rds/pdfs2/dpr15.pdf,
Eakin EG, Bull SS, Glasgow RE, Mason M: Reaching those most in need: a review of diabetes self-management interventions in disadvantaged populations. Diabetes Metab Res Rev. 2002, 18: 26-35. 10.1002/dmrr.266.
Tudor-Hart J: Semicontinuous screening of a whole community for hypertension. Lancet. 1970, 2: 223-227.
Brackertz N: Who is hard to reach and why?. ISR working paper. 2007, Available at: http://www.sisr.net/publications/0701brackertz.pdf
Pre-publication history
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/14/42/prepub
Acknowledgements
This research was supported by a Strategic Research Partnership Grant from the Cancer Council NSW to the Newcastle Cancer Control Collaborative and with infrastructure support from the Hunter Medical Research Institute (HMRI). BB is supported by a Cancer Institute NSW Career Development Fellowship. JB is supported by an Australian Research Council Post-Doctoral Industry Fellowship.
Author information
Authors and Affiliations
Corresponding author
Additional information
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
All authors conceived the study. MR conducted electronic searches. BB and MR coded articles. All authors contributed to drafting and finalising the manuscript. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
Bonevski, B., Randell, M., Paul, C. et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol 14, 42 (2014). https://doi.org/10.1186/1471-2288-14-42
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/1471-2288-14-42