School-based survey data collection is a common method of obtaining information about adolescent risk behaviors. Schools provide a setting where multiple students can be recruited simultaneously, and the school environment is conducive to paper and online surveys. However, school-based research has several limitations. First, schools receive numerous requests to participate in research and cannot accommodate them all (Esbensen et al., 2008; Plummer et al., 2014). Second, school-based research excludes students who are not in the classroom on the day of the survey, including those who are frequently truant, have dropped out of school, are attending alternative programs, or are home-schooled (Weitzman et al., 2003). Third, adolescents who engage in risky behavior and/or have chaotic family lives might be less likely to return parental consent forms, potentially biasing school-based samples (Anderman et al., 1995; Liu et al., 2017; Unger et al., 2004). Therefore, it is important to identify efficient and effective recruitment methods that can yield large, representative samples of adolescents.

Recruitment via social media could be an effective alternative to in-person recruitment in schools. Because most adolescents use social media, and because social media recruitment does not require in-person staff visits and the use of instructional time in school, social media could be a more efficient way to assemble large, representative samples for population-based studies. However, it is unclear whether social media recruitment produces samples that are representative of the underlying population and/or the school population.

Several studies have compared social media recruitment with other strategies to recruit adolescents. One study (Moreno et al., 2017) compared school- and clinic-based recruitment with social media recruitment for a physical activity monitoring study and found that in-person recruitment yielded more participants and was more efficient in terms of staff time and monetary expense per participant recruited. However, this was a high-intensity study that involved extensive physical activity monitoring. It is unclear whether findings would be similar for a brief survey. Another study (Gu et al., 2016) found that a mailed postcard with a QR code outperformed social media posts for recruiting rural adolescents. A study of sexual and gender minority youth (Stern et al., 2020) found that social media recruitment produced a more racially and ethnically diverse sample than the Youth Risk Behavior Surveillance Survey, which uses school-based recruitment. However, no known studies have compared the characteristics of adolescents recruited via social media with those of adolescents recruited in schools.

When the COVID-19 pandemic abruptly closed schools while we were in the process of recruiting a sample of adolescents, we switched from school-based recruitment to social media recruitment. We compared the demographic characteristics, mental health, and substance use of adolescents recruited via social media with those of adolescents recruited in schools. Because students who engage in risky behaviors and/or have mental health problems might be more likely to be absent from school and more likely to use social media (Vannucci et al., 2020; Vidal et al., 2020), we hypothesized that the adolescents recruited through social media would have a higher prevalence of mental health problems and substance use compared to those recruited in schools. We did not have a priori hypotheses about demographic differences in the two samples. We also compared the samples from both recruitment methods to the overall demographics of adolescents in the same state to determine which recruitment method produced the most representative sample.

Method

School-Based and Social Media Recruitment

The CalTeens project (Rogers et al., 2022) was designed to assess the association between proximity to cannabis retail outlets and cannabis use among California adolescents. We planned to recruit a representative statewide sample of California adolescents in high schools throughout the state. Details about the recruitment procedures are described by Rogers et al. (2022). In brief, we began by categorizing all California public high schools into nine strata based on their proximity to cannabis retailers (tertiles) and neighborhood socioeconomic status (tertiles). We were in the process of recruiting high schools in each of these nine strata in the fall of 2019 and the spring of 2020, when the COVID-19 pandemic closed all public schools in California in March of 2020. Therefore, we pivoted to a social media recruitment strategy. Targeted social media ads were placed on Facebook and Instagram using TrialPromoter (Reuter et al., 2016), a web-based tool that repeatedly posts ads on social media to recruit participants into clinical trials. The ads invited California teens to participate in a survey of attitudes and behaviors for a $20 gift card incentive. When social media users clicked on the ad, they were directed to a website to provide their contact information to obtain informed consent. If a parent of a teen clicked on an ad, they were directed to a website to provide parental consent and provide their child’s contact information so we could invite their child. We conducted crosschecks of names, contact information (including phone and address), and IP addresses to verify that participants lived in California and to prevent individuals from completing the survey more than once. Addresses were validated by searching on Google Maps to confirm that the address was a residence and by searching online reverse address databases such as Spokeo and PeopleFinders. We also contacted prospective participants and/or their parents by telephone to confirm their identity and address when necessary. Responses were excluded if respondents were not 14–17 years of age, did not provide a valid residential address, had the same IP address, email, or phone number as a previous participant, or provided a voice over internet protocol (VOIP) phone number rather than a valid cellular or landline phone number.

Informed Consent

Because the survey asked about sensitive topics and was not anonymous, our IRB required that we obtain parental consent and youth assent. For the school-based recruitment, we sent paper consent forms home with the students for their parents to sign. Students who obtained parental consent were invited to provide youth assent and participate in the survey. For the social media recruitment, when an adolescent clicked on the social media link, they arrived at a study website that asked for the adolescent’s and parent’s contact information. We emailed the parent with a link to the consent form. If the parent signed the consent form electronically, we emailed the student with a link to the assent form and survey.

Final Analytic Sample

In the school-based recruitment, 1297 students were approached in classrooms, 904 (70%) provided parental consent. Of those with parental consent, 819 (91%) also provided student assent. Of those, 737 (90%) completed the survey. Of the 1427 who engaged with the online recruitment (clicked on the ads and provided parental consent and student assent), 1180 (83%) completed the survey. Of those, 953 responses (80%) were verified and were included in the analytic sample. Unfortunately, we do not have information about people who clicked on the link but did not obtain consent, because our IRB required that we delete that information. The final sample was 1690 respondents (737 school-based and 953 online).

Statistical Analysis

We used t-tests and chi-squares to combine the school-based sample with the social media sample. Because the intraclass correlation of students nested within schools was very low (less than 0.05 for all variables) and because the social media participants were not clustered, we did not account for clustering in the analyses.

Results

Table 1 shows the comparison of the school sample (N = 737) and the social media sample (N = 953).

Table 1 Comparison of adolescents recruited in schools vs. via social media
Full size table

Demographics

The two samples did not differ significantly on age. The social media sample was overrepresented by boys (57%), whereas the school-based sample had a more balanced gender distribution. Compared to the school-based sample, the social media sample was overrepresented by whites and Asians and underrepresented by Hispanic/Latinx. The proportion of African Americans was similar across the two samples. The social media sample had a higher proportion of students who spoke only or mostly English at home. Figure 1 shows the racial/ethnic composition of (1) the samples produced by each recruitment method; (2) the population of California youth ages 0–17 in 2020, (3) the underlying population of the participating schools, and (4) a hypothetical sample composed of 50% school-recruited adolescents and 50% social media-recruited adolescents. This hypothetical combined sample yielded a racial/ethnic distribution that was more similar to the California population than either recruitment method alone. The school-based sample was very similar in racial/ethnic distribution to the underlying population of the participating schools.

Fig. 1
figure 1

Racial/ethnic distribution of students recruited via schools vs. social media, compared with the California population and population of the participating schools

Full size image

Mental Health and Substance Use

Compared to the school-based sample, the adolescents in the social media sample reported higher levels of depressive symptoms, anxiety symptoms, and perceived stress. The two samples did not differ significantly on past-month use of cannabis, tobacco, or alcohol.

Discussion

The abrupt closing of schools during the COVID-19 pandemic forced our research team to pivot from school-based data collection to social media data collection in the middle of an ongoing longitudinal study. Although changing data collection methods mid-study is not ideal, we were still able to recruit a large sample of adolescents that was approximately representative of the California population. In fact, a hypothetical sample composed of 50% school-recruited adolescents and 50% social media–recruited adolescents was more representative of the California population of adolescents than either recruitment method alone.

One drawback of the social media sample is that it recruited a smaller proportion of Hispanic/Latinx and students who spoke other languages at home, compared with school-based recruitment. Therefore, recruiting in public schools appears to be better for recruiting linguistically diverse samples.

It is interesting to note that although the two samples were different demographically, the adolescents’ self-reports of past-month substance use were similar. This suggests that social media recruitment could be a useful tool to estimate substance use prevalence, especially if samples could be weighted to represent the underlying population.

The social media sample reported more mental health symptoms (depression, anxiety, and stress) compared with the school sample. This is consistent with the finding that adolescents with mental health problems have more school absences and higher dropout rates (Allen et al., 2018; Esch et al., 2014). Adolescents with mental health problems also might have been diverted to alternative schools, which were not included in our school-based sample. Therefore, school-based samples are likely to miss the adolescents with the most severe mental health problems. However, it is also possible that the mental health problems reported by the social media-recruited students were a consequence of the COVID pandemic itself and the abrupt life changes that it caused just weeks before the adolescents were recruited. In addition, because adolescents with mental health problems are more likely to report heavy use of social media (Schønning et al., 2020), it is likely that the social media sample was overrepresented by adolescents with mental health problems.

Limitations

The school-based sample was recruited right before the COVID-19 pandemic, and the social media sample was recruited during the pandemic. It is unclear whether the start of the pandemic altered adolescents’ mental health or substance use patterns during this tumultuous time. It is possible that the higher prevalence of mental health problems among the social media sample was a consequence of the pandemic itself, because the pandemic adversely affected mental health among adolescents (Samji et al., 2022). The low proportions of adolescents in the social media sample who were Hispanic/Latinx or spoke another language at home might have been due to the fact that the social media ads appeared in English only. The school-based consent forms were printed in English and Spanish. Future studies should place social media ads in multiple languages to reflect the multiple languages spoken by the underlying population.

Although social media recruitment can be easier than recruiting and visiting schools, it also had drawbacks. A social media sample will likely be overrepresented by heavy social media users, who tend to have a higher prevalence of mental health problems than the general population of adolescents (Schønning et al., 2020). Staff had to validate every participant to make sure they were not bots or scammers, and it is possible that some ineligible participants managed to evade our validation efforts by misreporting their ages or creating multiple email accounts to earn multiple gift cards. Researchers who recruit via social media should be alert to these limitations.

Conclusion

Although school-based recruitment and social media recruitment produced samples with slightly different demographic characteristics, both samples yielded similar estimates of substance use prevalence. Social media recruitment could be an efficient strategy to recruit adolescents when school-based recruitment is not feasible. Social media recruitment could be an effective way to engage adolescents in research without disrupting schools, especially if the social media ads are targeted to more diverse audiences. A combination of school-based and social media recruitment methods could be used to overcome the selection biases inherent in each method. However, more research is needed to refine the procedures for obtaining representative samples via social media.