Background

Retention of study participants is vital to ensure the power and internal validity of longitudinal research [1–3]. A high attrition rate increases the risk of bias, particularly if those lost to follow-up differ from those retained in the study or if there is differential attrition between the intervention and control groups in a randomized controlled trial (RCT) [1].

Investigation to identify effective retention strategies has increased over recent years [4]. The following retention strategy themes were identified in two systematic reviews: (1) contact and scheduling methods, (2) visit characteristics, (3) study personnel, (4) nonfinancial incentives, (5) financial incentives, (6) reminders, (7) special tracking methods, (8) study description, (9) benefits of study, (10) reimbursement, (11) study identity, and (12) community involvement [4, 5]. Studies that used multiple retention strategies had higher retention rates [4]. Despite the knowledge gained from these systematic reviews, an inherent limitation is that they were limited to published data and did not allow for in-depth exploration of retention strategies and their implementation. They potentially overlooked other retention strategies or themes that may have been effective in existing research.

The objective of this study was to identify, via survey and in-depth, semi-structured interviews, cohort retention strategies and implementation approaches used in longitudinal clinical research studies that achieved high retention rates. We intentionally focused the research question on studies that had success with high retention rates, with the goal of learning from their experience and establishing a template and tools for successful retention practices. Findings from this study augment existing information on retention strategies from previous research and systematic reviews [3–9], and inform researchers designing studies and cohort retention tools.

Methods

A naturalistic inquiry approach was utilized to increase our understanding of retention practices in longitudinal studies that achieved high retention rates. The Johns Hopkins Medicine Institutional Review Board (IRB) database, which consists of the protocols from 7 separate IRBs with over 6500 active protocols, was queried to identify a convenience sample of studies that met the following inclusion criteria: 1) recruited ≥200 adult participants, 2) followed the participants for ≥1 year, and 3) retained ≥80% of participants using the following search: “Outcomes OR Cohort OR Longitudinal OR Long-term OR Long term OR Chronic OR Follow-up OR Follow AND Brain injury OR Chronic Obstructive Pulmonary Disease (COPD) OR Congestive Heart Failure (CHF) OR Hypertension OR “Kidney disease” OR Leukemia OR End-Stage Liver Disease (ESLD) OR Cancer OR Oncology OR Oncological OR Tumor OR End-Stage Renal Disease (ESRD) OR “Renal failure” OR Dialysis”.

Principal investigators of identified studies were contacted via an email with a description of this study and were asked to confirm study eligibility. For eligible studies, principal investigators were then asked for agreement to participate in this study and to provide contact information for a study team member who could answer detailed questions regarding the study’s retention strategies. In addition, investigators contacted members of their professional networks known to have conducted longitudinal studies within the institution. Six additional principal investigators were identified, four of whom were eligible and provided interviews. Of the 139 total studies identified through IRB query and related enquiries to established principal investigators, 22 met the inclusion criteria and 19 (86%) semi-structured interviews were completed. Reasons for not providing interviews included study completion >5 years prior, change in principal investigator, and departure of study staff responsible for retention efforts. Participants did not receive any financial incentive for participation. The Johns Hopkins Medicine Institutional Review Board approved this study with a waiver of informed consent.

Interview and data analysis procedures

The interview guide was created, reviewed and revised by four team members with expertise in longitudinal, patient-centered outcomes research (CDH, DMN, VDD, MNE). The interview guide included collection of background information on the eligible study’s design and methodology, followed by a detailed exploration of domains of inquiry related to retention strategies. Interviews were conducted in-person or by phone by two research assistants (MA, MC), who received training on the interview guide and data collection practices prior to initiation of the study. Interview questions were provided, upon request, via an online survey before the interviews to help the interviewee gather necessary information. Two study coordinators initially responded via the online survey (Active Surveillance of Prostate Cancer, and the Longitudinal Study of Alzheimer's Disease and Other Memory Disorders), then completed the remainder of interview by phone. Prior to the interviews, the research assistants performed an online search for any study-related publications and websites that provided information on the study aims, longitudinal research visits, retention strategies and retention rates to help focus the interview. In addition, study teams were asked to provide any additional study publications or written protocols that specifically outlined retention strategies. The team of investigators reviewed the data from each interview to explore emerging common findings. According to established qualitative research methods [10], the interview guide was iteratively refined and adapted to ask subsequent study participants about additional strategies that emerged in initial interviews. Then, the data was reviewed and independently coded by four investigators (MAA, MC, MNE, CDH), using a priori themes established in prior systematic reviews (Table 1) [4, 5]. After 19 interviews, data saturation was reached. A final review of the data was performed to synthesize overarching findings from the interviews.

Table 1 Retention strategy themes [4, 5]
Full size table

Results

Of the 19 participating studies, 13 were prospective cohort studies, 5 were RCTs, and 1 was a quasi-experimental study (Table 2). Included studies had diverse patient populations, including victims of intimate partner violence, persons living with human immunodeficiency virus (HIV) and acquired immunodeficiency syndrome (AIDS), survivors of critical illness and other serious illnesses. The target sample size for studies ranged from 255-10,000. Actual sample recruitment at the time of interview ranged from 205 for the Community Aging in Place – Advancing Better Living for Elders (CAPABLE) study [11] with older adults aging in place to 2528 for the Alzheimer’s Disease Anti-Inflammatory Prevention Trial (ADAPT) [12]. Though each of the studies retained at least 80% of participants, 10 (53%) of the 19 studies achieved >90% retention. Five of the participating studies had published articles describing the study protocol, including some reference to retention protocols [11, 13–16]. However, none of the studies empirically tested retention strategies within their study.

Table 2 Characteristics of included studies
Full size table

Table 3 provides a distribution of retention strategies employed across studies. The most commonly used strategies were from the following themes: reminders, visit characteristics, benefits of the study, and contact and scheduling. The strategy themes of study description and non-financial incentives were least reported by the participating studies. Some of the studies (Active Surveillance of Prostate Cancer study and Early Detection and Predicting Recurrence in Non Small Cell Lung Cancer study) embedded their research data collection in regular clinic visits to reduce patient burden, using the clinic protocols to serve as patient reminders. This group of clinic-based studies embedded within routine clinical care used very few strategies for recruitment and retention.

Table 3 Retention strategies used by each study
Full size table

Based on the interviews, the following were key findings associated with success in achieving high cohort retention rates: 1) cohort research staff are specialized, organized, persistent, and communicate well; 2) “personal touches” matter: tailoring retention strategies to individuals; and 3) written protocols do not reflect the many details of retention strategies employed during conduct of the study. Findings related to each of these three key messages are described below.

Research staff are specialized, organized, persistent, and communicate well

The majority of studies (n = 11) used a team approach to retain study participants, employing several staff members to collect data and manage retention. Two of these studies employed a full-time staff member dedicated to implementing retention strategies and focused on optimizing participant follow up. Five of 11 studies allocated a specific number of participants to one team member who served as the study’s primary point of contact for that group of participants. This person was given strategic support at team meetings, which included adapting or developing new approaches to overcome retention barriers when standard methods of contacting a participant had been exhausted. Other features of these successful teams included selecting appropriate individuals with responsibility for participant contact and providing intensive training and support on study protocols, including retention techniques. In recruiting research staff for cohort retention activities, study teams often screened applicants for experience, communication skills, cultural-competence and specialized knowledge of the population (e.g. Spanish-speaking, domestic violence advocate). One research team provided training focused on providing staff with a common understanding of empathy and sensitivity, including mock interviews to prepare new staff prior to contacting participants on their own [13, 17–19].

In most study teams, participant data tracking was complex; tracking methods were used to inform retention efforts and facilitate communication among team members. Rigorous monitoring, often using spreadsheets or databases, enabled team evaluation of the success of retention strategies or conversely the need to adapt or innovate for the most difficult-to-reach participants. Detailed records were kept for the participant and at least one additional contact person. These records were updated at every participant contact. Study staff used phone numbers, email, texting, social media (e.g., Facebook) and internet searches to locate participants. Studies visited participants’ homes, used personal delivery of reminders and searched obituaries, court documents and incarceration records to locate participants. One highly experienced study coordinator developed a checklist of techniques used to search for participants to help ensure a complete and systematic search process was consistently undertaken.

Study teams ensured regular internal communication regarding cohort retention issues through meetings involving research assistants, study coordinators, data managers and principal investigators. During these meetings, study teams examined their latest recruitment and retention rates, discussed strategies for participants who were difficult to contact and provided each other support and ideas. One study mentioned the importance of setting daily participant contact goals for each team member; another study used friendly competition as a way to engage team members.

“Personal touches” matter: tailoring retention strategies to individuals

To facilitate study visits, studies used tools and strategies tailored to the characteristics of individual participants or specific study requirements. For example, one study that required fasting prior to the visit provided snacks after the test was completed [20]. Some encouraged participants to bring items from home to make themselves comfortable, such as a blanket or pillow. Others provided meals or coffee. Two studies involving victims of intimate partner violence arranged for a safe meeting place and transportation as well as childcare [15, 21]. In many of the studies, parking and transportation were reimbursed. Studies also used financial compensation as a retention strategy, with amounts ranging from $10 for 1 h of survey response to $200 for a full day of data collection with invasive testing.

To help keep participants engaged throughout the study’s follow-up period, studies employed various methods. For instance, studies sent regular newsletters and cards for holidays, birthdays and even condolences. Many studies sent reminder letters about the study and upcoming visits at regular intervals. Lastly, many teams sought to develop strong staff-participant relationships by having the same staff throughout the follow-up period, which created long-standing relationships (e.g., decade-long) between study staff and participants. However, some studies, such as the CAPABLE study [11], did not require the same staff member to perform follow-up and yet were able to maintain a high retention rate throughout the study.

When appropriate, some studies sought to engage participants in groups, instead of individually, fostering a sense of community. For example, some studies hosted educational discussion forums where physicians or nurses led small group discussions related to disease management. Participants with Fuch’s corneal dystrophy were encouraged to invite the study team to family gatherings and reunions to provide education and testing for this genetic disorder. Other studies engaged participants through annual events, often held at entertainment venues (e.g., a dinner theater, horse racing) [20], using these events to encourage camaraderie among participants and study staff. Finally, one study invited participants to send in artwork created by the participant or their grandchildren to be used in future study correspondence and a calendar that featured different artwork for each month that was sent to each participant [20].

Protocols do not reflect the many details of retention strategies employed

Only 3 (16%) of 19 studies provided written cohort retention protocols. Recognizing this pattern after the initial interviews, we added interview questions to explore the lack of written protocols to delineate retention strategies. Interviews revealed that existing study protocols were not detailed regarding techniques to locate and retain participants or were out of date. Since strategies were frequently adapted to reflect new or revised retention strategies tailored to individual participant’s circumstances, with many of the changes verbally discussed during team meetings, revised retention strategies were noted but not updated in the actual protocol. Notably, principal investigators (PIs) and study coordinators agreed that IRB review and approval is an important process and that they developed study protocols recognizing IRB and ethical issues. In some studies, retention protocols submitted to the IRB included general principles for follow-up, rather than a detailed protocol, due to concern that a detailed protocol would require frequent IRB submissions with amendments and not be practicable to implement. Moreover, there were conflicting perceptions regarding whether or not the IRB would be supportive of cohort retention approaches that include internet or social media searches and persistence when trying to locate hard-to-find participants.

Discussion

The 19 studies with high cohort retention evaluated via in-depth interviews were heterogeneous in terms of design, sample size, target population and length of follow up. This heterogeneity allowed for exploration of a variety of approaches to cohort retention strategies and confirmed the 12 categories of retention strategies previously established. However, this study does provide further insight through the thematic analysis of in-depth interviews. These studies had research staff members who are specialized, persistent and collaborate well to accomplish their retention goals teams. They utilized personalized approaches and tailored retention strategies specific to participants in the study cohorts. This tailoring of strategies that are not formally documented in written form is an iterative process based largely on the experience and persistence of study coordinators. Cultural acceptability and feasibility of study and retention practices was evaluated by the expertise and experience of the study staff. This tailoring was facilitated by an understanding of social, cultural and environmental norms particular to the population studied.

Previous systematic reviews identified over 900 retention strategies across 88 studies, with higher retention attained in studies using more retention strategies [4, 5]. The present study suggests the importance of having a well-functioning research team that is organized, persistent, adaptable, and innovative. Prior studies focused on financial incentives to recruit and retain participants, finding a positive correlation between magnitude of incentive and retention rates [4, 5]. In our study, the use of financial incentives varied, yet all studies had high rates of retention, suggesting that participants also respond to additional strategies, such as facilitating visits and emphasizing other potential benefits of participating in the study. The in-depth interviews from this study confirm prior empirical analyses that demonstrated that achieving high retention rates requires use of multiple retention strategies and often many contact attempts by highly skilled staff [4, 22]. However, it is not clear which strategies are the most effective for study teams with limited budgets to implement. Though budget allocation for retention-specific activities was not assessed, most of these high retention studies were federally funded and adequately resourced; 15 of 19 studies had personnel that were trained and specialized, contributing to the high retention. Future studies should conduct comparative effectiveness evaluations regarding different budget allocations for retention to inform financial budget planning and prioritization among potential retention strategies.

While most research teams indicated that they have a retention protocol, after in-depth interviews, it became evident that the protocols were generally non-specific, often not reflecting all detailed aspects of retention practices. Moreover, the most successful study teams iteratively refined their retention processes in practice without updating protocols, and individualized them to specific participants. Maintaining updated retention protocols may be valuable for staff training and for maintaining high retention rates throughout the study, especially in the event of staff turnover. In addition, social media is in constant evolution and investigators should consider using technologies available to provide high quality longitudinal data. A recent article examined feasibility of using websites and social media for recruitment and demonstrated that a young female urban population is more likely to respond to these methods than traditional methods such as flyers and health fairs [23]. These recruitment strategies also use a small amount of study team resources, however it is unclear if this would be true for the purposes of retention.

Certain retention strategies are likely to have culturally-specific considerations. The question of appropriate methods and frequency for contact attempts is an important area for further exploration and may have variation in acceptability depending on the population. Likewise, the ethics of financial incentives have been widely discussed and remain an important ethical consideration. In addition, ethical regulation and oversight of studies has great variation within large academic centers, between institutions, and globally. In addition, IRB policies continue to evolve, particularly in response to new strategies such as digital and social media-based approaches for participant contact. Research practices and policies must ensure cultural-sensitivity and the protection of subjects while facilitating efficient and effective longitudinal research.

Findings from this study should be considered in light of several strengths and potential limitations. This study’s strength is the novelty of using in-depth interviews to obtain unpublished data from research teams to explore successful retention strategies for following large cohorts for at least one year. Moreover, a heterogeneous mix of high retention studies were included, with varying cohort sizes (205–2528 participants) and types of study population, including a national cohort study (ARDS Network Long Term Outcome Study – ALTOS) that enrolled over 900 acute respiratory distress syndrome (ARDS) survivors from more than 40 sites [17, 19]. However, this study also has potential limitations. First, only studies with ≥200 participants and retention rates ≥80% were considered; thereby, potentially limiting the comprehensiveness and generalizability of the data collected. However, all retention strategies from a prior systematic review [4] were exemplified through the interviews, offering some replication of findings. Future studies may examine how studies with low retention have used retention strategies and if practical resources can support study teams to improve retention rates. Also, in this study a convenience sample of studies were screened for eligibility from only one institution; hence additional retention strategies may be uncovered if conducted with a larger number of studies from other research centers. Social, cultural and regulatory body acceptance of the retention strategies described may vary by institution and internationally. As these retention strategies are used outside of the US, additional cultural tailoring of retention strategies may be required and should be studied empirically. Finally, the design of our study allowed us to identify strategies that were used together with a high degree of success. However, since these research teams applied multiple strategies we were not able to identify which individual strategies were superior. Given our insights regarding the need to tailor strategies to the context of the target population and other study factors, it is possible that each strategy identified could be implemented and yet retention goals still may not be met in a given study. To move the science of cohort retention forward, studies that compare retention strategies to identify best practices are needed.

There is need and ongoing effort to address the lack of attention and resources focused on cohort retention in longitudinal studies, including increasing number of studies reporting on retention strategies [4, 5]. To ensure validity and generalizability of results, future studies must consider and allocate resources for retention activities. This includes judiciously composing an effective research team specialized in implementing retention strategies. This project, undertaken as part of a National Heart, Lung, and Blood Institute-funded grant (R24HL111895) to create a national research infrastructure, aims to assist clinical researchers in achieving high retention rates by providing practical resources (see Tables 3, 4 and http://www.improvelto.com/).

Table 4 Overview of cohort retention toolkita
Full size table

Conclusions

In-depth interviews with research staff from longitudinal studies with high cohort retention are instructive in complimenting existing empirical literature in understanding successful strategies for cohort retention. Our findings emphasize the importance of creating and sustaining well-functioning research teams focused on cohort retention, applying personalized approaches and tailored retention strategies, and continually evaluating, refining and documenting retention strategies through the duration of the study.