Introduction

Public health researchers need to recruit and successfully enroll participants in many epidemiological studies. Individuals make participation decisions which can be influenced by many factors including how they were contacted, the topic of study, and cultural or social contexts [1, 2]. Researchers typically use multiple methods of contact to reach potential participants which includes postcards [3], email [4], and/or text message [5]. The methods used by researchers to contact individuals to elicit participation can influence the degree to which respondents represent the population of interest [6]. There is a growing body of evidence to suggest that multiple channels of contact using a range of modalities will improve participation in research studies [7, 8].

Postcard invitations have historically been used to successfully recruit participants and can be effective when used in conjunction with contemporary electronic and web-based enrollment methods [9]. Postcards are also effective in long term studies in assuring participant retention [10,11,12]. Telephoning potential participants is most effective with slightly older populations because direct interaction may be favored over other electronic means by some [13]. However, many potential participants lack land line telephones or may use caller ID to block unsolicited calls resulting in lower response rates and/or nonresponse bias [14]. Electronic means of contacting potential participants, including text message and email, is increasing [15]. While text messages and emails are effective in recruiting and reducing loss to follow-up, they can be used in a wide range of settings [16, 17]. Text messages and emails can also be used to address barriers which include lack of home internet access, travel time, and recruitment while being cognizant of protecting personal health information [18, 19]. Overall, multiple methods of contact and the number of contacts, when used in combination, can increase overall response rates. However, to maximize participation and minimize effort, researchers would benefit from predictors of response based on demographics, exposure, and other motivators for study participation [6, 15].

In the context of conducting a statewide random sample SARS-CoV-2 prevalence study that used multiple methods to contact participants, the purpose of the current study is to characterize the respondents whose participation followed either the postcard, phone outreach, or electronic means of invitation. We examine how the number of outreach attempts is related to participant characteristics. Specifically, we analyze the demographic characteristics, disease symptoms and experiences, overall health status, and reasons for participation in the study among respondents of each type and number of outreach attempts. Previous research has not examined symptomology and/or health status in understanding reasons for study participation. In the current analysis, we include these variables given that our data was collected at the early stages of an infectious disease pandemic where symptoms and health status were risk factors for morbidity and mortality. The current analysis will help provide an understanding as to participant response to different modalities of contact and attempts of contacts. We believe these findings can be useful for researchers who want to maximize the benefit of outreach methods that can influence the generalizability of their study.

Methods

The current analysis utilizes a cross-sectional study design to analyze survey data collected in April 2020 (See supplementary file) as part of a statewide population prevalence study of SARS-CoV-2 in Indiana. To better understand the data in the current study, we described the parent study [20] in which Indiana residents were randomly selected from among state residents who filed tax returns and had some contact information from other state databases. Participants who opted into the study, which took place early in the pandemic before vaccines were available, were PCR tested for active viral infection and simultaneously tested for antibodies indicating previous SARS-CoV-2 infection. Individuals who moved out of state, were less than 12 years old, were deceased, or institutionalized were excluded.

The parent study was publicly announced at a press conference with the Governor of Indiana and postcard notifications were sent to potential participants on April 24, 2020. Depending on available contact information in any state database (e.g., Bureau of Motor Vehicles, Medicaid, etc.) participants received a follow-up invitation by text/email 24 h after initial invitation, and then robocalls/texts 24 h after the initial text/emails. The robocalls/texts encouraged the selected individuals to opt into the study by registering either online or by calling a registration hotline and scheduling a date and time to get tested. All notifications and registrations were completed by close of business April 29, 2020, which resulted in 3,658 participants out of 15,495 invited [20]. The outcome of the study was participation in the prevalence study which was measured by getting tested after registering, which resulted in 99.2% of those who registered were tested.

Participants who registered for the study were tested at 68 facilities across the state and completed a short questionnaire online at time of registration, or when they arrived at a testing site, asking about health status, previous diagnoses, and reasons for participating in the study. All participants were also asked about any symptoms they were experiencing before being tested.

The main dependent variable in the current analysis is modality of contact that immediately preceded by participants’ registration for the study. If someone registered after receiving the postcard and before the text/email was sent, then they were considered as having been induced to participate by the postcard. The three groups of participants include those registering after [1] the postcard prompt [2], electronic text message and simultaneous email prompt (Text/Email); and [3] a robocall and simultaneous text message prompt (Robocall/Text). Because of variability in contact information available, not all respondents received all three methods of contact. As such, we created an additional variable measuring the unique number of initiated contacts to each participant. With this variable, we analyzed how the number of individual contacts was associated with participant characteristics.

The independent variables in our analysis were primarily from the registration survey and included demographics or other characteristics. Demographics include sex, age, race, ethnicity, and highest educational attainment. Other characteristics include reasons for participation in the study, self-reported health status, current Covid-19 symptoms, and previous exposure to Covid-19. Reasons for participation were measured on a 4-point Likert scale ranging from “not important” to “very important” and included the following items assessing potential personal, clinical, or societal benefits. Items measuring personal benefits included “feel good contributing to Covid-19 research,” “gaining knowledge about own Covid-19 status,” and “testing is free of charge.” Clinical benefits included “less risk of transmitting Covid-19 to family and friends” and societal benefits included “helping inform public health officials about Covid-19,” “contributing to scientific knowledge,” and “receiving support from family and friends.”

Descriptive statistics were calculated for the sample using frequencies and percentages. The analysis consisted of bivariate Chi square tests for each variable of interest by contact modality (e.g., postcard, text/email, robocall/text) and for the number of contacts for each participant. Variables that were statistically significant in bivariate analysis were included in a multinomial logistic regression to generate adjusted odds ratios and confidence intervals. All analyses were performed using SAS v.9.4 and p-values less than 0.05 were deemed statistically significant. The Institutional Review Board (IRB) at Indiana University deemed the study as not human subjects research under the public health surveillance exemption of the parent study.

Results

Of the 15,495 invited, the study sample included 3,658 individuals including 288 (7.9%) respondents that registered for the study after receiving a postcard, 968 (26.5%) who registered after receiving a text/email, and 2,402 (65.7%) that registered after receiving a robocall/text. There were 722 (19.7%) participants that registered after 1 contact; 1,753(47.9%) who registered after 2 contacts, and 1,183 (32.4%) who registered after receiving 3 contacts. Table 1 presents the demographic characteristics of respondents which differed by modality. Females made up 54.6% of the sample and responded at a higher rate for postcards (8.2% vs. 7.5%) and text/emails (28.1 vs. 24.6%) as compared to males (χ2 = 7.43, p = 0.025). In the same crosstabulation, males responded more frequently following robocalls/text at a higher rate than females (67.9% vs. 63.7%). No significant differences by modality were observed for race.

Table 1 Demographic characteristics of the sample by modality
Full size table

The age of participant was statistically different among the modalities. Specifically, those < 40 years responded at a higher rate for postcard, while those 40–59 years responded at a higher rate for text/email, and those 60 + responded at a higher rate for robocall/texts (χ2 = 20.1, p < 0.005). Education levels also differed by modality with those with the highest educational attainment responding most frequently to both postcards and text/emails than the other categories of education (χ2 = 61.8, p < 0.001). In the same crosstabulation, those with the lowest educational attainment responded at a higher rate for robocall/texts.

Differences in respondent health status and Covid-19 symptoms by modality are displayed in Table 2. Whereas there was no difference in self-reported health status by modality, significant differences by modality were observed by number of reported Covid-19 symptoms (χ2 = 50.1, p = < 0.001). Respondents reporting three or more symptoms responded to postcards and text/emails at the highest rates. Among those that reported no current symptoms from Covid-19, the highest percentage of participants responded after the robocalls/text notification.

Table 2 Health status and symptoms reported by the participant by modality
Full size table

Motivations for participating in the study are displayed in Table 3. Individuals responding following the robocalls/texts indicated that testing is free of charge was somewhat or very important in rates higher than respondents who participated following other modalities (χ2 = 7.87, p = 0.019). No other differences in reasons for participation were observed by modality. Those that responded following the robocalls/texts reported rates of living with someone who was previously positive for Covid-19 in the household at higher rates (χ2 = 7.07, p = 0.029). No other differences in respondent positivity rates were observed.

Table 3 Possible motivators for participating by modality
Full size table

The number of contacts that each participant received before registering for the study is displayed in Table 4. Compared to males, females responded at a higher percentage after 1 contact (21.4 vs. 17.9%, χ2 = 7.6, p = 0.023). With respect to age, those 60 + years of age responded by registering most often after 2 contacts (χ2 = 27.5, p < 0.001). The relationship between educational attainment and modality of response was also statistically significant. Specifically, those with the highest educational attainment, responded most frequently after 1 contact, while those with the lowest educational attainment responded after either 2 or 3 contacts most frequently (χ2 = 63.7, p < 0.001).

Table 4 Number of contacts by demographic and characteristics of participants
Full size table

When examining reasons for participation in the study, we observed a different rate of response by the number of contacts a participant received. Respondents that reported that feeling good contributing to research (χ2 = 11.3, p = 0.003) or gaining knowledge of their own Covid-19 status (χ2 = 6.5, p = 0.038) as somewhat or very important responded more frequently after 1 or 2 contacts. On the other hand, those that ranked either item as not important or barely important responded at the highest rates following 3 contacts. However, respondents that indicated that testing was free of charge was somewhat or very important responded most frequently after 3 contacts (χ2 = 9.2, p = 0.011). Lastly, participants who reported living with someone who was previously positive for Covid-19 in their household, most frequently respondent after 3 contacts (χ2 = 6.5, p = 0.039).

When analyzing the statistically significant independent variables in the logistic regression model, only sex, age, educational attainment, and a response that free testing was important were significantly related to modality the prompted response (see Table 5). Specifically, males were more likely than females to respond to text/emails as opposed to robocalls/texts (OR = 1.22, 95%CI 1.04–1.42). Compared to the youngest age group, those 60 years or older were less likely to respond to text/emails (OR = 0.94 95%CI 0.77–1.14) while more likely to respond to post cards (OR = 1.73, 95%CI 1.25–2.39). Compared to those with only a high school education, those with some college participated following post cards (OR = 0.37, 95%CI 0.25–0.55) less frequently than following robocalls/texts. Those with a college degree or higher responded to post cards (OR = 0.35, 95%CI 0.24–0.52) or text/emails (OR = 0.64, 95%CI 0.53–0.77) less frequently than to robocalls/texts.

Table 5 Multinomial logistic regression of modality of contact before registering (95% confidence intervals)
Full size table

Among those who had experienced any symptoms, motivations for participating were not significant in the model (p = 0.308). Individuals who indicated that free testing was an important motivator for participation were more likely to respond to text/emails, as compared to robocalls/texts (OR 1.27, 95%CI 1.06–1.53). No other motivating reason for participation was significant in the regression model.

Table 6 presents results from the regression analysis that examined the relationship between number of contacts and each of the independent variables that were statistically significant in bivariate analysis. Compared to the youngest age group, those 60 years and older were more likely to respond after 1 contact attempt (OR = 1.35, 95%CI 1.06–1.72). In the same model, compared to those with only a high school education, those with a college degree or higher were more likely to respond after a second contact (OR = 1.24, 95%CI 1.04–1.49) and less likely to respond after 1 contact (OR = 0.55 95%CI 0.42–0.72). Lastly, individuals who indicated that receiving free testing was a motivating reason to participate in the study, were more likely to respond after 1 contact (OR = 1.37, 95%CI 1.08–1.75).

Table 6 Multinomial logistic regression of number of contacts before registering (95% confidence intervals)
Full size table

Discussion

Successfully recruiting and enrolling participants in epidemiological studies is influenced by several factors, including how individuals are contacted, how often they are contacted, and other factors including cultural issues and demographic characteristics. In our regression analyses, males were more likely than females to respond following text/emails, as opposed to robocalls/texts. This nuance builds upon previous studies that characterize females as more likely to participate overall in health-related surveys [21, 22] by suggesting ways to increase male participation. Researchers wanting to assure adequate male representation in public health surveys may need to ensure that sufficient modalities are used, as opposed to just increased number of contacts.

In our study, those older than 60 years of age were more likely to respond following the postcard as compared robocalls/texts and were more likely to respond after only one contact. Given that advanced age was considered a significant risk factor for Covid-related morbidity and mortality, this finding is consistent with the context in which our data was collected. This oldest age group was also less likely to respond following text/email invitations relative to their youngest counterparts. Importantly, those in the youngest age group (less than 40 years) were also comprised of children aged 12–18 years of age whose email contact info on record with the state was likely that of their parents or guardians. Researchers who want to reach younger populations may need to do so through parental contact info if appropriate to the study’s aims, research design, and data availability. Previous studies have reported that those under 40 may be more difficult to recruit without multiple prompts [23]. This is consistent with our findings where the oldest age group was more likely than the youngest age group to respond after one invitation.

In our regression analyses, higher education attainment was associated with a greater likelihood of participation following robocalls/texts. In addition, higher levels of education were typically associated with needing two contacts, as opposed to three, to elicit participation. It is possible that those with higher education may have been in jobs that allowed more flexible and remote work schedules [24], thus enabling the ability to respond and participate sooner than those that participated following three prompts. Moreover, higher educational attainment may be associated with overall better health, health literacy, and awareness of risk from Covid-19 [25] which could have contributed to a less delayed participation in our study. Lastly, higher education may have been correlated with trust in governmental public health and research thus requiring slightly less prompting to participate [26].

Symptomology and previous experience with Covid-19 in the household was not associated with neither modality of invitation nor number of contacts. However, those indicating being motivated by “free testing” responded more frequently following texts/emails and following only one contact. Our data was collected at a time when access to Covid-19 tests were extremely limited in supply and were only available in emergency departments and inpatient settings. Having free access to both PCR and antibody tests was a motivator that appears to have induced earlier participation among many. This is important to note because while random selection attempts to achieve representative samples, our study highlights that participant-specific motivators still influence study engagement and is still a source of selection bias. Our findings can be put in the context of previous research that has reported how perceived exposure impacts the decision to get tested for disease associated research [27, 28]. Notably, our team has formally assessed non-response bias in previous analyses and reported minimal to none detected nonresponse bias using formal techniques [29].

Despite the new knowledge provided by our analysis, several limitations are worth noting. Findings should be interpreted within the context of the parent study which took place during the acute phase of an infectious disease pandemic. Moreover, the cross-sectional study design utilized in the current paper can only identify associations recognizing that our study is not suited for determining cause-and-effect. We recognize that of those who registered, they self-selected to participate in the study and were limited to those 12 years of age and older as well as non-institutionalized individuals in the population. These factors limit the generalizability of our findings. It should also be noted that based on contact information availability, not every participant received every modality of invitation to participate. There was a limitation in having to assign modality of participation based upon the timing of each invitation and when the individual agreed to participate in the study by registering. We acknowledge that, while helpful, this approach could have misclassified individuals who responded due to one modality and have not yet seen or were aware of a follow up modality (e.g., did not yet check email). Lastly, our study is limited by the inability to separate the effects of dual use of simultaneous modalities such as text/emails and robocall/texts.

In conclusion, recruiting and enrolling participants in primary epidemiological research is essential for public health research. Researchers can use multiple modalities and frequencies of contact to assure sufficient participation from diverse groups in their study. Our study highlighted how demographic and other characteristics are associated with participation following either postcard, text/email, or robocall/text outreach. Knowing who is most likely to respond following different contact approaches can help public health researchers assure that their study is not compromised by insufficient participation or poor external validity. This will aid researchers by allowing them to determine which modality of contact (postcards, texts, emails, robocalls) and the number of contacts that have the highest correlation of enrollment in research studies by demographic or other motivating characteristics.