Abstract
Background
It is well known that recruitment is a challenging aspect of any study involving human subjects. This challenge is exacerbated when the population sought is reticent to participate in research as is the case with pregnant women and individuals with depression. This paper compares recruitment methods used for the Food, Feelings, and Family Study, an observational, longitudinal pilot study concerning how diet and bisphenol A exposure affect maternal mood and cognitive function during and after pregnancy.
Methods
Pregnant women were recruited to this study over a period of 15 months using traditional methods, social media including paid and unpaid posts, and emails broadcast to the university community. Contingency analysis using the Pearson’s Chi-square test was used to determine if recruitment method was associated with likelihood of participation. T-tests were used to analyze Facebook advertisement success. ANOVAs and Fisher exact tests were used to determine if recruitment method was related to continuous and categorical demographics, respectively.
Results
Social media resulted in the largest number of recruits, followed by traditional methods and broadcast email. Women recruited through social media were less likely to participate. In contrast, use of broadcast email resulted in a smaller pool of recruits but these recruits were more likely to be eligible for and complete the study. Most women recruited via social media were the result of unpaid posts to the study’s Facebook page. Paid posts lasting at least 4 days were the most successful. Recruitment method was not associated with participant demographics.
Conclusions
Social media has the potential to recruit a large pool of potential subjects; however, when studies require a large time investment such as the case here, women recruited through social media are less likely to participate and complete the study than women recruited through other means.
Trial registration
N/A. This study does not describe a health care intervention.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Background
Pregnancy, a critical period in the life of many women, is often associated with mood disorders [1, 2]. There are gaps in the literature, in particular concerning the relationship between a woman’s diet and her mood and cognitive function during and following pregnancy. The gaps may exist because few pregnant women participate in research trials [3,4,5,6,7] and they have historically been classified as vulnerable human subjects because study participation could adversely affect pregnancy outcomes [4]. More recently, this categorization of pregnant women as a vulnerable population has been challenged, and viewed as an affront to women’s autonomy [8, 9]. When asked why they do not chose to participate in scientific research, pregnant women report that they: (i) lack interest in [6, 10] or distrust scientific research [9], (ii) feel as though preparing for the arrival of the baby leaves them with no time to participate [5, 6, 9], (iii) lack transportation to a study site [11, 12], (iv) experience disapproval from family and friends [5, 13], and (v) may be experiencing pregnancy-related health problems [5].
Compounding the problem, the symptoms of pregnancy-related depression, estimated to be present in up to 20% of women [1, 2], interfere with participation in research studies. Specifically, low motivation, lack of self-confidence, and perceived stigma are relevant barriers (systematically reviewed in [14]). In addition, the comorbidity of attention deficit disorders and depression [15] and the reported lack of attention due to pregnancy itself [16] may prevent the completion of tasks, potentially reducing the likelihood that pregnant women and particularly those with pregnancy-related depression will participate in long-term research activities. This adds to the difficulty inherent in recruiting for longitudinal studies as long-term time commitments deter participation [17, 18]. Nutrition-related studies also present unique barriers to participation [19, 20]. In particular, the collection of dietary intake data requires a substantial time commitment from the participant [21, 22]. In short, collectively, pregnancy, depression, longitudinal design, and issues surrounding dietary intake studies converge to make recruitment challenging, exhaust researcher efforts, delay the completion of studies, and impact methodological quality and study validity (reviewed in [12]). Of note, depending on the study design and participation criteria, 19 to 30% of clinical trials fail to meet enrollment targets [23, 24]. In light of these problems, it is imperative to understand which methods are the most effective to recruit and enroll challenging populations, such as pregnant women experiencing depression.
There are several approaches to recruit study participants, including traditional methods, social media, and email. Traditional recruitment methods include, but are not limited to, newspaper ads, posters, flyers, and word-of-mouth. Electronic recruitment via social media and email has the potential to have a broader reach than traditional methods. To wit, approximately 96 and 77% of US adults ages 18–29 years have a smartphone and home broadband, respectively [25, 26]. Further, social media recruitment methods typically involve services such as Facebook and Instagram, which are accessed by approximately 90% of adults in this age group [27].
The Food, Feelings, and Family (FFF) Study, launched in April 2018, was a pilot study aimed to recruit low income women experiencing pregnancy-related depression in their third trimester of pregnancy. Specifically, this longitudinal study was designed to investigate the effects of the antenatal diet and bisphenol A (BPA) exposure on ante- and postnatal depression and cognition. Because we encountered numerous barriers when attempting to recruit this understudied population, the purpose of this report is to share lessons learned to facilitate future research.
Methods
Overview of the food, feelings, and family study
The FFF Study recruited participants from April, 2018 to January 2020 (Fig. 1). In the original study design, eligible participants were to meet face-to-face with researchers on three occasions: during the third trimester of pregnancy (antenatal interview), 2–4 weeks postpartum, and 12 weeks postpartum. During these meetings, participants were to perform various tasks such as providing blood and urine samples and completing demographic, health, depression, and anxiety surveys, a diet history questionnaire, and a cognitive test. For the purposes of this report, “recruitment” refers to the number of individuals who completed the screening survey while “participation” reflects completion of the entire study.
Study timeline versus number of screening surveys completed. Black, white, and gray bars indicate the number of participants that completed the screening survey as a result of traditional recruitment practices, social media, and broadcast email, respectively, as described in the Methods section. Social media refers to both Facebook and Instagram. Alterations to the study protocol to increase participation are indicated by arrows. Letters on the x-axis indicate the first letter of each month of the year, starting with April represented by the letter A
Initially, the study aimed to recruit 144 low-income pregnant women, half of which were experiencing pregnancy-related depression as assessed by the Edinburgh Postpartum Depression Scale (EPDS). The EPDS is a ten-question self-rating scale developed to assess depressive symptoms in the postnatal period [28]. This scale can also be used to identify depression in the antenatal period [29,30,31]. Successful recruitment would yield a final sample of 100 based on an anticipated attrition rate of approximately 30% [32]. After 6 months of unsuccessful recruitment, however, we eliminated the requirements for biological specimen collection, EPDS scores reflective of pregnancy-related depression, as well as income restriction, in hopes of attracting more potential participants (Fig. 1). In addition, 4 months later, when the Psychology Experiment Building Language (PEBL), the cognitive testing software used for this study, became available online for both Windows and Macintosh operating systems, we were able to eliminate the need for face-to-face interviews and thereby engage participants online, allowing us to reach a national audience. Finally, 1 year after recruitment began, the third interview was eliminated to further motivate participation and retention.
The final inclusion criteria were: (1) ages 18–35 years, (2) singleton pregnancies and (3) 28–37 week’s gestation at the antenatal interview. The exclusion criteria were: (1) use of brain training games (e.g. Lumosity), (2) multifetal pregnancies, (3) diagnosis of mental illness other than perinatal and major depression, and (4) lack of internet access. Women that were interested in our study were invited to complete the eligibility screening survey, hosted in Qualtrics, which included questions about demographics, medical history, use of brain training games, and the EPDS. Brain training games mimic the cognitive tests used for this study and have potential to confound the interpretation of cognitive data [33,34,35].
Participation in the FFF Study required two interviews and the completion of several assessments. Women were invited to participate face-to-face or online. Face-to-face participants were provided with a password protected computer dedicated to the project to complete the interviews. Participants who opted to participate online were required to have a webcam, microphone, and computer. The antenatal interview at 28–37 weeks gestation lasted approximately 1 h during which time participants completed demographic, mood [28, 36, 37] and sleep surveys [38], the Diet History Questionnaire III [39], cognitive failures questionnaires [40], and cognitive tests [41,42,43]. Between the ante- and postnatal interviews, participants were asked to submit the BPA Exposure Assessment Module [44] and complete three 24-h dietary recalls via the Automated Self-Administered 24-Hour Dietary Assessment Tool [45]. The postnatal interview, held 2–4 weeks following birth, was approximately 30–45 min in length, and required participants to complete surveys regarding mood [28, 36, 37], sleep [38], and cognitive failures [40], and undergo cognitive testing [41,42,43].
Participants were incentivized with $100 in Amazon gift cards distributed after completion of various tasks related to the study. In addition, following the antenatal interview participants received “belly butter.” Further, following completion of the postnatal interview, participants were given a baby bib with the FFF Study logo and a personalized dietary analysis. These gifts were mailed to online participants. All study aspects were approved by the Institutional Review Board of Texas State University, protocol #4897. Written informed consent was obtained from all subjects.
Traditional recruitment procedures
From the onset of the FFF Study we employed a comprehensive approach to traditional recruitment throughout the Central Texas area, spanning from San Antonio to Pflugerville, TX and covering four counties. For example, flyers advertising the FFF Study (Fig. 2a), including the logo, study purpose, basic eligibility requirements, incentives offered, and contact information were distributed to healthcare facilities and businesses catering to pregnant women or parents of young children, such as obstetrician offices, birthing centers, public health clinics, child care centers, childrens clothing stores, and libraries. Flyers were also posted across the university campus and in community centers and churches. In addition, booths were hosted at medical centers, low income housing facilities, and food banks. Also, free classes related to infant nutrition and feeding healthful complementary foods to young children were offered on the university campus and at various facilities. Finally, researchers visited approximately 50 potential recruitment sites within a 50-mile radius of their campus such as obstetrician-gynecologists, midwives, doulas, and a large hospital network in central Texas to establish relationships with these stakeholders. Thereafter, these stakeholders were visited at least once each month and suppled with additional recruitment flyers (Fig. 2a) and monthly newsletters (Additional Files 1 and 2) either on paper or via email for distribution to potential participants as per their preference. Incentives were not offered to these stakeholders. The newsletters included topics likely to be of interest to pregnant women, such as how to find credible sources of information related to medicine and science (Additional File 1), child nutrition (Additional File 2), lactation, and pregnancy massage. The hospital network presented information regarding the FFF Study at a meeting of their obstetrician-gynecologists and distributed the flyers and monthly newsletters to obstetrician-gynecologists within their network for use in participant referrals. Despite these efforts, few women completed the screening survey: therefore, we decided to expand our efforts to include recruitment via social media and broadcast emails.
Sample recruitment flyer and example of a Facebook paid promotion. The recruitment flyer (a) contains the Food, Feelings, and Family logo and contact information (university logo is not shown due to copyright) and information regarding the study’s objective, participation expectations, incentives, eligibility criteria, and investigator information. The Facebook promotion (b) displays the study’s logo and objective, a link to the screening survey, contact phone number, and text indicating where emojis depicting fruits and vegetables were located. All images other than the university logo (a) and the emojis (b) were original or in the public domain
Social media recruitment procedures
A Facebook page and Instagram account entitled “Food, Feelings, Family” and displaying the study logo, a link to the study’s webpage, a description of the research team, and contact information were created early in the study (Fig. 1). A social media influencer was consulted to provide insight regarding how to build a Facebook and Instagram following. Based on her recommendations, the researcher organizing the FFF page asked to join Facebook groups likely to have expectant mothers or women experiencing perinatal depression as members (Additional File 3). Terms used to identify potential groups included: pregnancy, pregnant, mother, mom, mommy, perinatal, depression, and anxiety. For private groups, researchers contacted the group administrator and asked permission to join. In addition, health care providers that were located in Central Texas were contacted via Facebook and asked to share the FFF Study recruitment flyer on their pages. Instagram was used to follow the accounts of healthcare professionals and pregnancy related entities. In return, these accounts could then follow the FFF Study Instagram and share our information to their feed.
Strategies aimed at leveraging the Facebook page to recruit participants included (i) sharing of posts to FFF Study researchers’ personal Facebook pages, (ii) shared posts to Facebook groups, and (iii) paid Facebook/Instagram promotions. Utilizing graphic design platforms, posts related to the study in general or postpartum depression in particular were created. There was a total of 73 posts published on the FFF Facebook page. The 59 unpaid posts provided general information about our study, perinatal depression, the FFF research team, opportunities to attend free infant feeding classes, and date-specific salutations (e.g. “The FFF team wishes you a Happy 4th of July!”). The 14 paid posts included (1) a stock image or icon (e.g. pregnant mother, mom and baby, food icons), (2) an engaging quote or brief text (e.g. “Share with any mommy-to-be!”), (3) information about our study, and (4) a link to our screening survey (Fig. 2b). The paid Facebook posts were promoted from April to November 2019. The Facebook Ads manager allows the user to set a post objective (e.g. increase page and post awareness) and target audience (e.g. gender, age, and interests). This allowed paid posts to be targeted to Facebook users that were female, ages 18–50, and residing within the US. Posts were promoted for 1–7 days, at a cost of approximately $5 per day. Paid posts had a set budget and Facebook performed automatic bidding to achieve the greatest number of clicks for the lowest cost. All paid promotions were shared to Facebook mobile, Facebook desktop, and Instagram news feeds. Facebook or Instagram users that viewed our paid promotions could opt to visit our study page, click the link to our screening survey, like, comment, or share our posts.
Broadcast email recruitment
In addition to social media recruitment, a broadcast email was sent to approximately 4000 university faculty and staff on a monthly basis between April and November, 2019 (Fig. 1). Each email included: (i) the study’s goals, (ii) inclusion criteria, (iii) incentives, (iv) ways to participate (e.g. in person or online), (v) a brief overview of what participation would entail, and (vi) researchers’ contact information. Recipients of the email were encouraged to share study information with potential participants.
Data collection and statistical analyses
All recruits indicated how they learned about the FFF Study in the online screening survey. Contingency tables using the Pearson’s Chi-squared test were used to determine if recruitment method was associated with likelihood to complete the study. A P-value < 0.05 indicated significance for the overall comparison (e.g. recruitment method x eligibility). Post-hoc tests for significant comparisons were conducted by calculating Z-scores (the standardized adjusted residual) for each combination of recruitment method and the categorical variable of interest. Z-scores were converted to estimated P-values by squaring each Z-score then calculating the right-tailed probability using the Chi-square distribution with one degree of freedom. The resulting P-value was compared to the adjusted significance value identified using the Bonferroni correction to reduce the likelihood of Type I error resulting from multiple comparisons (P = 0.008).
The Facebook Ads Manager provided data for each paid advertisement, including the number of days the promotion ran, post impressions, post engagement, and cost per click. Post impressions quantify the number of times our post was delivered to a user’s Facebook newsfeed. Post engagement reflects the number of times an action such as a like, share, or comment was made on each post. The insights tab on the FFF Facebook page provided data regarding the total number of posts on the FFF Facebook feed as well as paid reach. Paid reach specifies the number of Facebook users that had a paid post appear on their screens. With the exception of cost per click, the effects of paid post duration and subject on these metrics were compared using a Welch’s t-test for unequal variances. Differences in cost per click were determined via Student’s t-test. These data are displayed as mean ± SD in the text.
Demographic information was self-reported by participants during the antenatal interview. An ANOVA was used to determine if age, income, gestational age, and EPDS score differed by recruitment method. Due to the relatively small sample size and expected cell counts, Fisher Exact Tests were used to determine if the recruitment method was associated with ethnicity, education, poverty, and pregnancy number. All analyses were performed using IBM SPSS Statistics for Macintosh, Version 24.
Results
Recruitment and enrollment
The traditional recruitment method utilized during the first 11 months of the FFF Study yielded only 14 completed online screening surveys (Fig. 1). Also, during this time, 13 individuals completed screening surveys due to visiting our Facebook page or Instagram feed. Responses to our screening survey increased dramatically in April 2019 following the implementation of paid social media promotions on Facebook and broadcasting emails to the university faculty and staff.
Altogether, 172 individuals began the FFF screening survey; of these, 167 completed it, and were considered “recruited” (Fig. 3). The majority of individuals were recruited via social media (n = 88), including Facebook and Instagram, followed by traditional methods (n = 51), and broadcast email (n = 28). Of these, 12 women recruited via social media, 16 via traditional methods, and 14 via broadcast email completed the study. Interestingly, the likelihood of a recruit to complete the study differed by recruitment method (P = 0.0003). Specifically, fewer women recruited via social media (n = 12; P = 0.0003) and more women recruited via broadcast email (n = 14; P = 0.0009) than expected completed the study. There was also a difference in eligibility between recruitment methods (P = 0.007); women recruited via email (P = 0.008) were more likely to be eligible for the study. Forty-one, 54.5, and 21.4% of women recruited using traditional methods, social media, and broadcast email were not eligible to participate in the study, respectively. Reasons for ineligibility are shown in Fig. 3. The primary reason was lack of access to a computer with a microphone or a webcam necessary for online participation (n = 24). When only eligible recruits were considered, the likelihood of completing the study differed by recruitment method (P = 0.023). In summary, while social media resulted in the largest number of completed screening surveys, broadcast email, while resulting in a lower number of completed screening surveys overall, produced a pool of recruits that were more likely to be eligible for the study.
Flowchart of participation by recruitment methods from screening to study completion. The term “Social Media” includes both Facebook and Instagram. Diagonal arrows indicate the number of participants that changed from in person to online interviews and vice versa. Percentages represent the percent of total recruits and may not add up to 100 due to rounding
Facebook metrics
The FFF Facebook page, active for 58 weeks, received 225 views, 102 likes, and 111 followers. As mentioned above, prior to using paid posts to promote our study on Facebook, 13 women that completed the screening survey indicated they heard about the FFF Study on social media (Fig. 1). After we increased the visibility of the FFF Facebook page in March 2019 by joining or contacting other Facebook pages (Additional File 3), 27 individuals completed the screening survey. Our first paid Facebook promotion on April 30, 2019 corresponded to Maternal Mental Health Awareness Week [46]. This promotion was followed by 13 additional paid and promoted Facebook posts over a period of 6 months. Thirty-nine women completed screening surveys as a result of paid Facebook promotions between April 30 and November 17, 2019 (Fig. 1). The total cost of these promotions was approximately $265 (Table 1). Post success was measured by greater paid reach, impressions, and engagement, and lower cost per click. The success of posts was related to post duration but not subject (i.e. FFF Study or postpartum symptoms or statistics). Specifically, promotion of a post for at least 4 days, as opposed to 3 or fewer days, resulted in greater paid reach (2501 ± 1300 vs 239 ± 79, P = 0.001), a higher number of impressions (2679 ± 1354 vs 242 ± 81, P = 0.001), and more engagement (366 ± 194 vs 53.2 ± 20, P = 0.001). Cost per click did not differ by post duration (P = 0.143). Interestingly, 15 of the women that completed the screening survey indicated they found out about the FFF Study on social media after termination of paid Facebook promotions (Fig. 1).
Recruit location and participant demographics
Screening survey respondents included women across the US residing in 28 out of the 50 states (Fig. 4a). Despite recruiting nationwide, most respondents lived in TX (n = 112; Fig. 4b). In fact, the majority respondents were from the Central Texas area (Fig. 4c), including the cities of Austin (n = 37), San Marcos (n = 29), Kyle (n = 7), Buda (n = 5), and New Braunfels (n = 4). With respect to participants, only 13 lived outside of the Central Texas area.
Geographical distribution of recruits and participants in the Food, Feelings, and Family Study. Google maps was used to indicate locations across (a) the United States, (b) Texas, and (c) Central Texas. Red circles indicate women that completed the screening survey (recruits). Green circles display the location of study participants. In (C) blue triangles indicate the location of cities and towns in Central Texas. The three recruits that completed the screening survey and lived outside the US are not included. The map of the United States was modified from the original [47]. The maps of Texas and Central Texas were derived from [48]. These maps are available under the GNU General Public License
Recruitment method was not related to participant age, income, gestational age, or EPDS score at screening (Table 2). In addition, the EPDS scores were not reflective of pregnancy-related depression. There were no differences in ethnicity, education, poverty, or pregnancy number due to recruitment method. All participants recruited via the broadcast email had completed college or some graduate/professional school with income above the poverty level. Women recruited through traditional methods and broadcast email were mostly primiparous.
Discussion
The FFF was an ambitious pilot study, aimed at addressing important gaps in the literature related to diet, BPA exposure, mood, and cognition during and after pregnancy. Importantly, this study is unique because we focus on how the mother’s diet affects her mental state, whereas most studies concern how maternal diet impacts child-related outcomes (e.g. [49,50,51]). Despite extensive efforts over a time span of almost 2 years, we were unable to reach our goal of recruiting 144 participants. In the current report, we compare three recruitment strategies: traditional, social media, and broadcast email in an attempt to determine which was the most successful at reaching this elusive population. We found that while social media posts generated the greatest number of completed screening surveys, these women were less likely to enroll and participate in the study.
Pregnant women report that they do not participate in research studies because they often lack interest in or distrust scientific research [6, 9, 10], experience disapproval from friends and family [3, 5], and feel as though they do not have time to participate [5, 9]. On the other hand, pregnant women indicate they are motivated to participate when they receive personal benefits, including the potential for improved pregnancy outcome [52, 53], health education [54], and improvements to their own health [10, 52]. Altruistic reasons, such as helping future patients [53] and contributing to knowledge, also spur participation in research studies [9, 10, 52, 54,55,56]. For example, Daniels et al. [55] found that 93% of pregnant women were motivated to participate in a study because they would contribute to science. Our social media posts included text encouraging women to “help researchers” and “prevent postpartum depression.” While altruism may have temporarily inspired women to complete the screening surveys, this motivation perhaps waned when it was time to actually participate.
The original study design included the collection of biological fluids (blood and urine) to corroborate dietary intake and BPA exposure data. Other researchers have noted that pregnant women are reticent to provide biological specimens [56, 57]. The findings of this previous work are supported in the current study by the increased number of completed screening surveys once biospecimen collection was removed from the study protocol.
The fact that traditional recruitment methods yielded the highest number of participants is not surprising because previous research has found that personable, enthusiastic, non-judgmental, and empathetic recruitment personnel and research staff aid in recruitment efforts [6, 58]. Others have shown that recruitment was also improved by working with obstetricians for participant referrals [3, 12, 53, 54], as in the current study. Although Gatny and Axinn [54] suggest that health education may motivate women to participate, we were unable to successfully recruit potential participants by offering free early childhood nutrition and feeding classes. While it is time consuming to travel to businesses and medical offices, meet with office managers, and distribute study materials, this personal touch made a difference and, combined with word-of-mouth recruitment, proved worth the investment.
A personal connection may also explain our success using institutional broadcast emails. The primary benefit to the broadcast email approach is that it took little time compared to the traditional approach, yet yielded a similar participation rate. When considering the entire sample, women recruited via broadcast email were more likely to be eligible for the study. Importantly, women that completed the screening survey as a result of traditional recruitment methods or broadcast email were more likely to actually participate in the study. Other studies report employing broadcast emails but do not analyze this category separately, generally grouping email along with social media into “electronic” or “online” recruitment [59]. We chose to examine broadcast email as a separate category because the large number of individuals reached at our institution (approximately 4000) by a single email was greater than the paid reach of all but one of our paid Facebook posts.
While the literature is somewhat discordant regarding the use of social media for recruitment with studies showing greater, similar, and lower efficacy than traditional methods, a pattern emerges when the time commitment on the part of the participant is considered. We hypothesize that social media recruitment results in a large number of participants when studies require relatively little time to complete. For example, social media is an excellent recruitment tool when study participation requires very little of the participants’ time to complete a survey that can be reached with one click on a Facebook post [60, 61]. To wit, Admon et al. [61] recruited 5 times more individuals using Facebook than in-clinic recruitment to participate in a survey-based study requiring only 15 min to complete. Similarly, another study showed the enormous reach of social media by recruiting 1075 women in early pregnancy over an 18-week period to respond to an online survey, 10–15 min in duration, regarding childbirth preferences [60]. The screening survey for the FFF Study required only 10–15 min to complete and could be reached via one click from our Facebook posts. Reflective of this, social media was the most effective recruitment method, responsible for over half of the women that completed the screening survey, but a different story emerges with respect to participation. Participation in the FFF Study entailed at least 2 h of the participants’ time spread over multiple days to complete. Importantly, women recruited via social media were less likely to participate than women recruited via traditional methods or via broadcast email. While it is not possible to know why eligible women failed to participate in the study, we can speculate. For example, we offered an incentive of $100, for our subjects with relatively high income, this amount of money may not have been sufficient to secure commitment to the study. Alternatively, women may have lacked time.
High interest, followed by lower participation rates for potential participants found through social media, termed “conscientious recruits” by Frandsen [62], is emerging as a common problem when studies require a longer time commitment, for example randomized controlled trials examining prevention of weight gain [63] and smoking cessation drug efficacy [62]. Adam et al. [64] recruited almost 10 times the number of pregnant women per day using social media than through traditional recruitment methods similar to those employed in the current study, but also found that women recruited via social media were less likely to participate in their trial comparing counseling to standard care for management of appropriate pregnancy-related weight gain. Similarly, Christensen et al., [65] recruited women trying to conceive to a nutrition and lifestyle study. They used offline and online methods to recruit, including Facebook ads. While they conclude that online recruitment was more successful than offline, only 3.6% of their actual participants were recruited via Facebook advertisements. Finally, Van Gelder et al., [66] reported greater success with traditional methods than paid Facebook promotions regarding recruitment of pregnant women into their prospective, long-term longitudinal study. They also report, similar to the current study, lower participation rates among women recruited through Facebook vs traditional methods. In summary, recruitment via social media can be successful when only a short period of attention is required to complete the study, as with survey-based research. When considering the larger time commitments required of randomized controlled trials or longitudinal studies, recruitment through social media may be detrimental in terms of both time and financial investments.
To our surprise, most of the women recruited through social media were not a result of paid promotions. Rather, these women completed the screening survey because they saw our unpaid posts on groups they were members of. At a cost of almost $7 per completed screening survey, the cost effectiveness of paid social media promotions is questionable. In contrast to our results, Bennetts et al., [67] reported far greater success with paid posts than unpaid. Their success may be due to the fact that the first part of their study involved a short time commitment, so participants were likely to click the link and complete this survey portion of the study when using Facebook. Importantly, when contacted to complete a second, follow-up survey, Bennetts et al., [67] note that participation fell by almost half, supporting our hypothesis that social media is most useful for short, one-time only survey-based research.
Previous studies have reported that participants recruited through social media tend to be white [68,69,70], more educated [69, 71,72,73], and affluent [74] when compared to a reference population. Given the small sample size of the current study, a comparison to a reference population is not instructive. Within the current study, these parameters did not differ by recruitment method.
As mentioned above, women with perinatal depression are difficult to recruit to research studies (systematically reviewed in [14]); therefore, the EPDS scores of participants at the time of recruitment should be considered. Scores of 13 or greater on the EPDS have traditionally been thought to indicate risk for pregnancy-related depression [28]. More recently, in recognition of the fact that depression is not binary, the EPDS score has been used in a more continuous manner, with increasing scores corresponding to a greater likelihood of depression [75]. In this new system, EPDS scores of 0 to 6 are thought to reflect no or minimal depression while scores of 7 to 13 are indicative of mild depression. Collectively, our participants would be categorized as having minimal depression based on their responses to the EPDS survey administered during screening. Subsequent publications from our group will explore how EPDS scores change over time and relate to diet in this cohort.
While informative, this study had several limitations. Firstly, our group had no prior experience with paid Facebook promotions. Although we consulted with a social media expert, with additional experience, the posts could possibly have been improved to better target pregnant women, however this would have required market research which was beyond the scope of this pilot study. Secondly, the sample size was small for two primary reasons revolving around technology access and the time commitment required to complete the study. Specifically, among women recruited via social media and email, the most common reason for ineligibility was lack of access to a computer with a webcam and microphone. This barrier highlights the importance of adapting studies, when possible, to accommodate participation through a more ubiquitous device, such as a smartphone. For the present study, it was not possible to use a smartphone because our cognitive test platform, PEBL, required the use of a Windows or Macintosh operating system. Finally, although not a limitation, the relatively large amount of time participants must devote to longitudinal studies is a challenge to recruitment that should be considered.
Conclusions
Social media has the potential to reach an enormous number of individuals, particularly in hard to recruit populations such as women experiencing pregnancy-related depression. Unfortunately, for women recruited through social media, follow through resulting in full participation in the FFF study was low. In short, recruitment though social media may not be the best recruitment method for studies involving a relatively large time commitment. Rather, a personal connection, for example between study personnel and healthcare professionals, as with traditional recruitment methods, or institutional loyalty, as seen following broadcast email, promotes participation particularly in studies requiring a large time commitment.
Availability of data and materials
Data may be made available by the corresponding author upon a reasonable request.
Abbreviations
- FFF:
-
Food, feelings, and family
- BPA:
-
Bisphenol A
- EPDS:
-
Edinburgh postpartum depression survey
- PEBL:
-
Psychology experiment building language
References
Underwood L, Waldie K, D'Souza S, Peterson ER, Morton S. A review of longitudinal studies on antenatal and postnatal depression. Arch Womens Ment Health. 2016;19:711–20.
Dadi AF, Miller ER, Bisetegn TA, Mwanri L. Global burden of antenatal depression and its association with adverse birth outcomes: an umbrella review. BMC Public Health. 2020;20:173.
Frew PM, Saint-Victor DS, Isaacs MB, Kim S, Swamy GK, Sheffield JS, et al. Recruitment and retention of pregnant women into clinical research trials: an overview of challenges, facilitators, and best practices. Clin Infect Dis. 2014;59(Suppl 7):S400–7.
Blehar MC, Spong C, Grady C, Goldkind SF, Sahin L, Clayton JA. Enrolling pregnant women: issues in clinical research. Womens Health Issues. 2013;23:e39–45.
van Delft K, Schwertner-Tiepelmann N, Thakar R, Sultan AH. Recruitment of pregnant women in research. J Obstet Gynaecol. 2013;33:442–6.
Muggli E, Curd H, Nagle C, Forster D, Halliday J. Engaging pregnant women in observational research: a qualitative exploratory study. BMC Pregnancy Childbirth. 2018;18:334.
Jacobsen TN, Nohr EA, Frydenberg M. Selection by socioeconomic factors into the Danish National Birth Cohort. Eur J Epidemiol. 2010;25:349–55.
Krubiner CB, Faden RR. Pregnant women should not be categorised as a 'vulnerable population' in biomedical research studies: ending a vicious cycle of 'vulnerability'. J Med Ethics. 2017;43:664–5.
van der Zande ISE, van der Graaf R, Oudijk MA, van Delden JJM. Vulnerability of pregnant women in clinical research. J Med Ethics. 2017;43:657–63.
Andrighetti HJ, Semaka A, Austin JC. Women's experiences of participating in a prospective, longitudinal postpartum depression study: insights for perinatal mental health researchers. J Med Ethics. 2017;20:547–59.
Infanti JJ, O'Dea A, Gibson I, McGuire BE, Newell J, Glynn LG, et al. Reasons for participation and non-participation in a diabetes prevention trial among women with prior gestational diabetes mellitus (GDM). BMC Med Res Methodol. 2014;14:13.
Tooher RL, Middleton PF, Crowther CA. A thematic analysis of factors influencing recruitment to maternal and perinatal trials. BMC Pregnancy Childbirth. 2008;8:36.
Frew PM, Saint-Victor DS, Owens LE, Omer SB. Socioecological and message framing factors influencing maternal influenza immunization among minority women. Vaccine. 2014;32:1736–44.
Hughes-Morley A, Young B, Waheed W, Small N, Bower P. Factors affecting recruitment into depression trials: systematic review, meta-synthesis and conceptual framework. J Affect Disord. 2015;172:274–90.
Chen MH, Pan TL, Hsu JW, Huang KL, Su TP, Li CT, et al. Attention-deficit hyperactivity disorder comorbidity and antidepressant resistance among patients with major depression: a nationwide longitudinal study. Eur Neuropsychopharmacol. 2016;26:1760–7.
de Groot RH, Adam JJ, Hornstra G. Selective attention deficits during human pregnancy. Neurosci Lett. 2003;340:21–4.
Panjari M, Bell R, Adams J, Morrow C, Papalia MA, Astbury J, et al. Methodology and challenges to recruitment to a randomized, double-blind, placebo-controlled trial of oral DHEA in postmenopausal women. J Womens Health. 2002;17:1559–65.
Blumenthal DS, Sung J, Coates R, Williams J, Liff J. Recruitment and retention of subjects for a longitudinal cancer prevention study in an inner-city black community. Health Serv Res. 1995;30:197–205.
Hure AJ, Smith R, Collins CE. A recruiting failure turned success. BMC Health Serv Res. 2008;8:64.
Di Noia J, Schultz S, Monica D. Recruitment and retention of WIC participants in a longitudinal dietary intervention trial. Contemp Clin Trials Commun. 2019;16:100438.
Naska A, Lagiou A, Lagiou P. Dietary assessment methods in epidemiological research: current state of the art and future prospects. F1000Res. 2017;6:926.
Shim JS, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health. 2014;36:e2014009.
Carlisle B, Kimmelman J, Ramsay T, MacKinnon N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clin Trials. 2015;12:77–83.
Kitterman DR, Cheng SK, Dilts DM, Orwoll ES. The prevalence and economic impact of low-enrolling clinical studies at an academic medical center. Acad Med. 2011;86:1360–6.
Mobile Fact Sheet. https://www.pewresearch.org/internet/fact-sheet/mobile/2020. Accessed 26 June 2020.
Internet/Broadband Fact Sheet 2020. https://www.pewresearch.org/internet/fact-sheet/internet-broadband/. Accessed 26 June 2020.
Social Media Fact Sheet. https://www.pewresearch.org/internet/fact-sheet/social-media/. Accessed 26 June 2020.
Cox JL, Holden JM, Sagovsky R. Detection of postnatal depression. Development of the 10-item Edinburgh postnatal depression scale. Br J Psychiatry. 1987;150:782–6.
Chorwe-Sungani G, Chipps J. A systematic review of screening instruments for depression for use in antenatal services in low resource settings. BMC Psychiatry. 2017;17:112.
van Heyningen T, Honikman S, Tomlinson M, Field S, Myer L. Comparison of mental health screening tools for detecting antenatal depression and anxiety disorders in south African women. PLoS One. 2018;13:e0193697.
Milgrom J, Gemmill AW, Bilszta JL, Hayes B, Barnett B, Brooks J, et al. Antenatal risk factors for postnatal depression: a large prospective study. J Affect Disord. 2008;108:147–57.
Duffy J, Rolph R, Gale C, Hirsch M, Khan KS, Ziebland S, et al. Core outcome sets in women's and newborn health: a systematic review. BJOG. 2017;124:1481–9.
Al-Thaqib A, Al-Sultan F, Al-Zahrani A, Al-Kahtani F, Al-Regaiey K, Iqbal M, et al. Brain training games enhance cognitive function in healthy subjects. Med Sci Monit Basic Res. 2018;24:63–9.
Nouchi R, Taki Y, Takeuchi H, Hashizume H, Nozawa T, Kambara T, et al. Brain training game boosts executive functions, working memory and processing speed in the young adults: a randomized controlled trial. PLoS One. 2013;8:e55518.
Choi E, Shin SH, Ryu JK, Jung KI, Kim SY, Park MH. Commercial video games and cognitive functions: video game genres and modulating factors of cognitive enhancement. Behav Brain Funct. 2020;16:2.
Somerville S, Dedman K, Hagan R, Oxnam E, Wettinger M, Byrne S, et al. The perinatal anxiety screening scale: development and preliminary validation. Arch Womens Ment Health. 2014;17:443–54.
Condon JT. The assessment of antenatal emotional attachment: development of a questionnaire instrument. Br J Med Psychol. 1993;66(Pt 2):167–83.
Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC. Quantification of sleepiness: a new approach. Psychophysiology. 1973;10:431–6.
Diet History Questionnaire III. https://epi.grants.cancer.gov/dhq3/. Accessed 26 June 2020.
Broadbent DE, Cooper PF, FitzGerald P, Parkes KR. The cognitive failures questionnaire (CFQ) and its correlates. Br J Clin Psychol. 1982;21:1–16.
Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol. 1935;18:643–62.
Dinges DF, Powell JW. Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behav Res Methods Instrum Comput. 1985;17:652–5.
Englund CE, Reeves DL, Shingledecker CA, Thorne DR, Wilson KP, Hegge FW. Unified tri-service cognitive performance assessment battery (UTC-PAB) I. Design and specification of the battery. Naval Health Res Center Rep. 1987:87–10. https://www.researchgate.net/publication/235145083_Unified_Tri-Service_Cognitive_Performance_Assessment_Battery_UTCPAB_1_Design_and_Specification_of_the_Battery.
Nomura SO, Harnack L, Robien K. Estimating bisphenol a exposure levels using a questionnaire targeting known sources of exposure. Public Health Nutr. 2016;19:593–606.
Subar AF, Kirkpatrick SI, Mittl B, Zimmerman TP, Thompson FE, Bingley C, et al. The automated self-administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet. 2012;112:1134–7.
Maternal Mental Health Awareness Week. https://www.thebluedotproject.org/mmhweek20192019. Accessed 26 June 2020.
Wikimedia Commons. https://commons.wikimedia.org/wiki/File:Blank_map_of_the_United_States.PNG. Accessed 07 August 2020.
Alexrk, Wikimedia Commons. https://commons.wikimedia.org/wiki/File:USA_Texas_location_map.svg#/media/File:USA_Texas_location_map.svg. Accessed 07 August 2020.
Pina-Camacho L, Jensen SK, Gaysina D, Barker ED. Maternal depression symptoms, unhealthy diet and child emotional-behavioural dysregulation. Psychol Med. 2015;45:1851–60.
Barker ED, Kirkham N, Ng J, Jensen SK. Prenatal maternal depression symptoms and nutrition, and child cognitive function. Br J Psychiatry. 2013;203:417–21.
Hibbeln JR, Davis JM, Steer C, Emmett P, Rogers I, Williams C, et al. Maternal seafood consumption in pregnancy and neurodevelopmental outcomes in childhood (ALSPAC study): an observational cohort study. Lancet. 2007;369:578–85.
Rodger MA, Makropoulos D, Walker M, Keely E, Karovitch A, Wells PS. Participation of pregnant women in clinical trials: will they participate and why? Am J Perinatol. 2003;20:69–76.
Kenyon S, Dixon-Woods M, Jackson CJ, Windridge K, Pitchforth E. Participating in a trial in a critical situation: a qualitative study in pregnancy. Qual Saf Health Care. 2006;15:98–101.
Gatny HH, Axinn WG. Willingness to participate in research during pregnancy: race, experience, and motivation. Field Methods. 2011;24:135–54.
Daniels JL, Savitz DA, Bradley C, Dole N, Evenson KR, Eucker B, et al. Attitudes toward participation in a pregnancy and child cohort study. Paediatr Perinat Epidemiol. 2006;20:260–6.
Joseph JW, Neidich AB, Ober C, Ross LF. Empirical data about women's attitudes toward a biobank focused on pregnancy outcomes. Am J Med Genet A. 2008;146a:305–11.
Nechuta S, Mudd LM, Elliott MR, Lepkowski JM, Paneth N. Attitudes of pregnant women towards collection of biological specimens during pregnancy and at birth. Paediatr Perinat Epidemiol. 2012;26:272–5.
Zielinski R, Ackerson K, Misiunas RB, Miller JM. Feasibility of a longitudinal study of women anticipating first pregnancy and assessed by multiple pelvic exams: recruitment and retention challenges. Contemp Clin Trials. 2010;31:544–8.
Shere M, Zhao XY, Koren G. The role of social media in recruiting for clinical trials in pregnancy. PLoS One. 2014;9:e92744.
Arcia A. Facebook advertisements for inexpensive participant recruitment among women in early pregnancy. Health Educ Behav. 2014;41:237–41.
Admon L, Haefner JK, Kolenic GE, Chang T, Davis MM, Moniz MH. Recruiting pregnant patients for survey research: a Head to Head comparison of social media-based versus clinic-based approaches. J Med Internet Res. 2016;18:e326.
Frandsen M, Thow M, Ferguson SG. The effectiveness of social media (Facebook) compared with more traditional advertising methods for recruiting eligible participants to Health Research studies: a randomized, controlled clinical trila. JMIR Res Protoc. 2016;5:e161.
Partridge SR, Balestracci K, Wong AT, Hebden L, McGeechan K, Denney-Wilson E, et al. Effective strategies to recruit Young adults into the TXT2BFiT mHealth randomized controlled trial for weight gain prevention. JMIR Res Protoc. 2015;4:e66.
Adam LM, Manca DP, Bell RC. Can Facebook be used for research? Experiences Using Facebook to Recruit Pregnant Women for a Randomized Controlled Trial. J Med Internet Res. 2016;18:e250.
Christensen T, Riis AH, Hatch EE, Wise LA, Nielsen MG, Rothman KJ, et al. Costs and efficiency of online and offline recruitment methods: a web-based cohort study. J Med Internet Res. 2017;19:e58.
van Gelder M, van de Belt TH, Engelen L, Hooijer R, Bredie SJH, Roeleveld N. Google AdWords and Facebook ads for recruitment of pregnant women into a prospective cohort study with long-term follow-up. Matern Child Health J. 2019;23:1285–91.
Bennetts SK, Hokke S, Crawford S, Hackworth NJ, Leach LS, Nguyen C, et al. Using paid and free Facebook methods to recruit Australian parents to an online survey: an evaluation. J Med Internet Res. 2019;21:e11206.
Head BF, Dean E, Flanigan T, Swicegood J, Keating MD. Advertising for cognitive interviews:a comparison of Facebook, craigslist, and snowball recruiting. Soc Sci Comput Rev. 2016;34:360–77.
Burrell ER, Pines HA, Robbie E, Coleman L, Murphy RD, Hess KL, et al. Use of the location-based social networking application GRINDR as a recruitment tool in rectal microbicide development research. AIDS Behav. 2012;16:1816–20.
Carter-Harris L, Bartlett Ellis R, Warrick A, Rawl S. Beyond traditional newspaper advertisement: leveraging Facebook-targeted advertisement to recruit long-term smokers for research. J Med Internet Res. 2016;18:e117.
Heffner JL, Wyszynski CM, Comstock B, Mercer LD, Bricker J. Overcoming recruitment challenges of web-based interventions for tobacco use: the case of web-based acceptance and commitment therapy for smoking cessation. Addict Behav. 2013;38:2473–6.
Quach S, Pereira JA, Russell ML, Wormsbecker AE, Ramsay H, Crowe L, et al. The good, bad, and ugly of online recruitment of parents for health-related focus groups: lessons learned. J Med Internet Res. 2013;15:e250.
Hernandez-Romieu AC, Sullivan PS, Sanchez TH, Kelley CF, Peterson JL, Del Rio C, et al. The comparability of men who have sex with men recruited from venue-time-space sampling and facebook: a cohort study. JMIR Res Protoc. 2014;3:e37.
Balfe M, Doyle F, Conroy R. Using Facebook to recruit young adults for qualitative research projects: how difficult is it? Comput Inform Nurs. 2012;30:511–5.
McCabe-Beane JE, Segre LS, Perkhounkova Y, Stuart S, O’Hara MW. The identification of severity ranges for the Edinburgh postnatal depression scale. Reprod Infant Psychol. 2016;34:293–303.
Acknowledgements
The authors wish to thank the numerous undergraduate and graduate students without whose help this work would not have been completed, particularly Tessa Comstock, Erika Gonzalez, Andrea Alvarez and JD Muraida. The authors also thank Vickie Howell for her advice and insight regarding social media and Dr. Nick Bishop for his assistance with the statistical analyses.
Funding
This study was supported by the Texas State University Research Enhancement Program. The funding body played no role in the design of the study, the collection, analysis, and interpretation of data, nor in writing the manuscript.
Author information
Authors and Affiliations
Contributions
RS and CA recruited and interviewed the participants, compiled and analyzed data, and wrote the draft of this manuscript. SC and ML conceived of the study, obtained funding, oversaw the study, and edited the manuscript to its final form. All authors have read and approved this manuscript.
Corresponding author
Ethics declarations
Ethics and approval to participate
Human subjects protocols were approved by the Institutional Review Board of Texas State University, Federal wide Assurance of Compliance # FWA00000191. Written informed consent was obtained from each subject.
Consent for publication
not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1:.
January 2019 FFF Study Newsletter. sample newsletter sent to stakeholders and participants.
Additional file 2:.
February 2019 FFF Study Newsletter. sample newsletter sent to stakeholders and participants.
Additional file 3: Table S1.
Facebook groups and pages joined or contacted to promote FFF Study. list of Facebook groups.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Smith, R., Alvarez, C., Crixell, S. et al. The Food, Feelings, and Family Study: comparison of the efficacy of traditional methods, social media, and broadcast email to recruit pregnant women to an observational, longitudinal nutrition study. BMC Pregnancy Childbirth 21, 203 (2021). https://doi.org/10.1186/s12884-021-03680-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12884-021-03680-1