To test these expectations, I rely on two survey data sets—one from Lucid and one from the ANES—as well as evidence from a separate survey experiment and its replication.Footnote 1 This section provides a description of these different data sources, the measures used, and the methodological approaches. An overview can be found in Table 1.
Regarding the first data set, I fielded a demographically diverse survey of U.S. adults (N = 811) in mid-August 2019 using Lucid’s Fulcrum Academic service (referred to as “Lucid”). Though not nationally representative, Lucid targets representativeness on several known demographic benchmarks—including race, age, sex, household income, and Census region. Coppock and McClellan (2019) find that demographic and experimental findings on Lucid are similar to those of U.S. national benchmarks. Lucid has been used for survey data collection in previous studies on similar topics (Callaghan et al., 2019; Lunz Trujillo et al., 2021). I also account for potential deviations from representativeness by using post-stratification weights matched to population benchmarks on race, age, sex, income, and educational attainment. More information can be found in section A of the supplement.
I also use data from the 2019 ANES Pilot study (“”) to test the above hypotheses. The pilot was conducted using non-probability sampling and was fielded on American adults online through YouGov—an online opt-in panel—in December 2019 by taking a random sample of individuals from a nationally representative pool (according to 2016 Census data) (N = 3165). Since this was an online opt-in panel, to further approximate national representativeness, YouGov used propensity score matching to get a sample as close to Census targets as possible and ended up with a sample size of 3165 respondents. YouGov also developed weights according to respondent race, age, gender, education level, and 2016 presidential vote choice (ANES, 2019); more information can be found in Table A4 in the supplemental materials. This method produces a sample that looks similar to a probability sample on the matched characteristics but may still differ in unknown ways on unmatched characteristics.
The first variable I operationalize is rural residency. There are various ways to define and measure this term, and decades of academic research by sociologists, demographers, geographers, and other scholars have debated the best way to do so (such as Hart et al., 2005; Miller & Luloff, 1981; Nemerever & Rogers, 2021). Researchers often use definitions and quantitative scales created by government agencies based on objective criteria, such as population density, population size, and distance from metropolitan centers. These include measurements such as the Rural–Urban Commuting Area Codes (“RUCA”), which were developed by the USDA and the University of Washington. The 10 basic RUCA codes, which can be divided into 33 more granular codes, are based on Census tracts. These codes are defined according to whether an area is a metropolitan center or small-town core (according to population density), along with the percent of the population that commutes to such cores. I use RUCA codes as one measure of living in a rural area in the Lucid data. To translate the RUCA codes into an urban/rural dichotomous measure, I collapse the 10-code measure into metro (designations 1–3) and non-metro (designations 4–10), as this is one way to capture rural versus non-rural using these codes (see Nemerever & Rogers, 2021, p. 274).
However, the source of the RUCA codes at the University of Washington and the USDA recommends using another categorization of urban versus rural using the more granular 33 RUCA codes (RUCA Rural Health Research Center, n.d.). This alternative classification strategy (“Categorization A”) is detailed in the supplemental materials. For this reason, I analyze the data using both transformations of the RUCA codes. The alternative coding schema does not yield notably different results in the present study. I also measure rural residency based on whether the respondent said that they currently live in a rural area, or if they grew up in a rural area.
The above hypotheses suggest that this RUCA measure of rural residency should not significantly correspond to anti-intellectualism. However, rural social identification should. To measure rural identity, and its counterpart, metro identity, I adapt the partisan identity strength scale by Huddy and colleagues (Huddy, 2001, 2013; Huddy et al., 2015) for the Lucid survey. These studies argue that the political effects of social identities are most pronounced among stronger identifiers, so measuring the strength of the identity is key in understanding the nature and degree of the identity’s impact on outcomes of interest. The supplemental materials list the question wordings for the five-item battery used to measure rural (and metro) identity strength. Respondents were only given the rural identity strength questions if they said that they either grew up in a rural area, or if they currently live in a rural area (N = 486). All other respondents were given the metro identity strength questions (N = 324). The scale reliability coefficient of the five items is 0.90 (0.89 for the metro identity strength items). Figure A2 in the supplement shows the distribution of this measure in the Lucid data. Table 3 below shows the cross-tabulation of rural identity and rural residency for the ANES data.
Rural residency in the ANES data is measured according to whether the respondent said that they currently live in a rural area or small town, based on Munis (2020). Rural identity is measured according to one of two versions. The first version of the question asks respondents, “Regardless of where you currently live, do you usually think of yourself as a city person, a suburb person, a small-town person, a country (or rural) person, or something else?”, while the second version of the question asks, “Regardless of where you currently live, where do you feel you belong or fit in the best: cities, suburbs, small towns, or the countryside (rural areas)?” If the respondent selected either version’s small town or rural/country response, they are coded as a rural identifier. This differs from the Lucid study question, as it measures rural identity using only one question. However, it still captures the difference between how someone feels or identifies—getting at the psychological dimension—versus the categorization or objective measurement of the rural residency question.
The anti-intellectualism measure comes from Oliver and Rahn’s (2016) scale, which is used for the Lucid and ANES studies, and in the experiment. The scale is divided into two parts: anti-intellectualism and anti-elitism. This anti-intellectualism subdimension of the populism scale has been used to measure anti-intellectualism in previous work (such as Motta, 2018) and employs three items. The first asks how much respondents put trust in the wisdom of ordinary people rather than in experts. The second asks how much respondents agreed that, when it comes to really important questions, scientific facts do not help very much. Finally, respondents indicate how much they agree that ordinary people can really use the help of experts to understand complicated things like science (reverse-coded). Response options for all three are a seven-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree”. The scale reliability coefficient of these three measures is 0.44 for the Lucid data and 0.71 for the ANES data. Page 10 of the supplemental materials shows the distribution of this measure.
The regression analyses also include demographic control measures that could account for factors driving the link between rural identification and anti-intellectualism, based on previous research. These include political and religious variables that have been found to correlate with both rurality and anti-intellectualism, specifically party identity strength (7-point scale), symbolic ideology (7-point) (Gimpel et al., 2020; Motta, 2018), and evangelical or born-again Christian (binary with evangelical or born-again Christian = 1) (Childs & Melton, 1983; Claussen, 2004). I also include standard demographic control variables like age (continuous variable), gender (binary with female = 1), race/ethnicity (a binary variable for Black and another for Hispanic), education level (seven-point scale), and household income (24-point scale). The ANES models also include respondent region (Northeast, South, Midwest, and West) with the base region being the Northeast. The distribution for the anti-intellectualism variable (which follows a normal distribution) can be found in the supplemental materials on pages 10 and 19. The correlations of all variables used can be found on pages 9 and 17 of the appendix.
The Lucid and ANES data sets provide correlational tests in support of the hypotheses, but they do not provide evidence for the implied causality of rural identification leading to anti-intellectual sentiment. Existing studies have found causal evidence that manipulating the level of anti-intellectualism in information provided to respondents affects opposition to areas of expert consensus (Merkley, 2020). The theory discussed above by SIT suggests the in-group psychological attachment forms first and then out-group affect forms (Branscombe et al., 1999; Huddy, 2003). To back up this theoretically assumed causal relationship, I experimentally manipulate rural salience for those who self-identify as rural and use the anti-intellectualism scale as the dependent variable.
In early August 2020 and again in December 2020 as validation, I conducted a survey experiment using Lucid. This experiment, while also using Lucid, was run separately from the Lucid survey data described above from August 2019. The initial experiment was conducted in August 2020. Here, 360 individuals consented to take the survey; 334 remained after adding in RUCA codes and excluding respondents who did not answer the key variables in the analyses. Information on the demographics of the respondents can be found in Table A7 of the supplement. Respondents were asked what best describes the community that they grew up in. Those who said “rural” were coded as rural residents. Then, respondents were asked what best describes the community they currently live in. Those who said “rural” here were also coded as rural residents. For the second validation experiment, respondents were also given the options of “rural” or “small town” for current or grew up location; these rural and small town respondents are both considered rural here.
All respondents were assigned to one of two experimental conditions: the control condition (“Control”) or the treatment condition (“Treatment”). Respondents in the treatment condition (N = 164) first received a message saying that they would read an excerpt from a local newspaper on the following screen. Then, respondents viewed a picture of a rural landscape, and were presented with a short 121-word paragraph highlighting the benefits of living in a rural area. This paragraph also talks about how many younger people who grew up in rural areas are moving back. This treatment is meant to increase in-group salience through making participants generally more aware of rurality and by highlighting positive in-group characteristics to increase in-group pride. At any point in time, people hold various social identities. Whichever social identities are relevant at any given moment depends on the salience and strength of that identity (Huddy, 2003). If a particular identity is more salient, then any values or associations with out-groups should also be heightened. Though this treatment should not affect those who are not rural social identifiers, the overall mean of the treatment group should increase for those exposed to the treatment, as the increase will be driven by the individuals who are the rural identifiers.
Respondents in the treatment group then answered the dependent variable. Those respondents who were assigned to the control condition (N = 170) did not receive any treatment and only answered the dependent variable. The outcome variable is the anti-intellectualism scale by Oliver and Rahn (2016) measured in the same way the above analyses. The scale reliability coefficient of these three measures is 0.42, with an average interitem covariance of 0.02. See page 21 of the supplemental materials for details and specific treatment wording.
To verify these experimental results, I also fielded a replication study. This replication study was conducted via Lucid in December 2020 (N = 495). The experiment is essentially identical to the initial experiment. After randomly assigning respondents to one of the two conditions, 237 were assigned to the above-described treatment meant to increase rural salience. The remaining 258 were assigned to the control condition and did not receive any text. All respondents were given two sets of three questions each to measure anti-intellectualism—the outcome variable—as well as the five-item rural identity strength battery. The rural identity strength here serves as a manipulation check to verify whether the treatment increased rural identity strength. The rural identity versus anti-intellectualism scale order was randomly presented. The rural social identity strength scale has a Cronbach’s alpha of 0.90. The anti-intellectualism scale has a Cronbach’s alpha of 0.54. Since this is fairly low, I repeat all analyses using the two-question anti-intellectualism scale, which has a Cronbach’s alpha of 0.71.