Public opinion polls in the 2020 presidential election suffered from errors of “unusual magnitude,” the highest in 40 years for surveys estimating the national popular vote and in at least 20 years for state-level polls, according to a study conducted by the American Association for Public Opinion Research. (Baltz 2021, emphasis added)

This quote ran in a Washington Post article under the headline, “2020 presidential polls suffered worst performance in decades…” This article is just one example of the recent concern among polling professionals, politicians, media, and academics about the accuracy of polling. Concern about polling problems is not new, but high profile polling misses—e.g., the 2015 UK general election, the Brexit vote in 2016, and the French presidential election of 2017 (Prosser and Mellon 2018) have heighted attention to polling issues. In fact, the American Association for Public Opinion Research (AAPOR, the organization mentioned in the opening quote and a leader among organizations investigating polling problems) formed special committees and commissioned detailed reports following both the 2016 and 2020 U.S. presidential elections. While 2016 was a clear miss (Kennedy et al. 2018), the quote and headline mentioned here belie more optimistic assessments of 2020—including Clinton et al.’s (2021) assessment that, “Most national polls accurately estimated that President Joe Biden would get more votes than President Donald Trump nationally.” At the same time, one problem continues to plague political polls and seems to be getting worse: Democratic bias.

Although 2020 seemed to alleviate polling problems, insofar as few predicted the wrong winner, the average amount polls overstated Democrats’ vote totals increased (Clinton et al. 2021; Clinton et al. 2022; Panagopoulos 2021). And the problem is not going away, notes Nate Cohn (2022), chief political analyst for the New York Times:

Early in the 2020 cycle, we noticed that Joe Biden seemed to be outperforming Mrs. Clinton in the same places where the polls overestimated her four years earlier. That pattern didn’t necessarily mean the polls would be wrong—it could have just reflected Mr. Biden’s promised strength among white working-class voters, for instance—but it was a warning sign. That warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.

Polling problems come in many flavors and from many sources. The 2016 AAPOR report focuses on the following four: nonresponse bias and deficient weighting, late deciding, misspecified likely voter models, and, the focus of this paper, the Shy Trump Voter Hypothesis. Of these, the report concludes nonresponse bias, deficient weighting (especially weighting to correct overrepresentation of college graduates), and late deciding (a swing toward Trump in the final days of the campaign) account for the lion’s share of inaccuracies. The 2020 report revisits these problems and concludes that pollsters corrected several problems from 2016—especially weighting problems—and finds fewer problems with late deciders in 2020. However, in 2020 nonresponse is still highlighted:

Some explanations of polling error can be ruled out according to the patterns found in the polls, but identifying conclusively why polls overstated the Democratic-Republican margin relative to the certified vote appears to be impossible with the available data. Reliable information is lacking on the demographics, opinions, and vote choice of those not included in polls (either because they were excluded from the sampling frame or else because they chose not to participate), making it impossible to compare voters who responded to polls with voters who did not (Clinton et al. 2021).

These conclusions are largely verified by Panagopoulos (2021). However, Panagopoulos diverges when it comes to late deciders in 2020 and shy Trump voters in both elections. Both AAPOR reports downplay shy Trump voters as a major source of bias, but faced with evidence of worsening bias that systematically favors Democrats, Panagopoulos concludes additional research is merited. This article delves deeper than previous studies into the types of pressures voters feel when reporting which presidential candidate they support.

Where prior work on the Shy Trump Voter Hypothesis has assessed whether voters feel pressure to hide support for Trump in the aggregate, no prior work has identified which individuals feel which type of pressure (because some also feel pressure to hide support for Clinton in 2016 and Biden in 2020). Identifying which individuals feel which type of pressure is incredibly important. Imagine an electorate in which all Democrats feel pressure to say they will support the Democrat and all Republicans feel pressure to say they will support the Republican. There could be aggregate differences in which candidate voters feel pressure to say they support (for example, maybe Democrats outnumber Republicans), but these types of pressures for these types of individuals would not produce any shy voters. In this scenario, social pressures would reinforce partisan predispositions and voters would have few incentives to prevaricate in surveys (say they will vote for one candidate in the survey, and then vote for the other in the voting booth)—aggregate imbalance in social pressure notwithstanding. On the other hand, if there are some Democrats who feel pressure to say they will support the Republican and vice versa, the possibility of shy voters affecting polling accuracy gains steam. Research already suggests that individuals in heterogeneous core networks make different voting decisions than individuals in homogeneous core networks—potentially due to different social pressures (Sokhey and McClurg 2012; Ryan 2010). In similar fashion, the possibility of shy voters cannot be dismissed without examining which type of pressure voters feel.

In what follows, we revisit scholarship on social desirability bias—the more general phenomenon behind the Shy Trump Voter Hypothesis—and describe our study. To summarize, we alter an experiment often called the fake-good/fake-bad experiment in which voters are randomly assigned to report the candidate they would say they would vote for if they wanted to make the best(worst) impression on others (Holbrook et al. 2003; Klar and Krupnikov 2016). We too randomize which version voters get (best or worst impression), but then we return to each voter with the opposite condition. Doing so enables us to measure which type of pressure each voter feels. This individual-level information allows us to tabulate potential shy voters in a new way and to investigate the causes and consequences of the social pressures voters feel.

Notably, we find both shy Trump and shy Biden voters which likely reduces aggregate bias produced by shy voters. We cannot make a definitive conclusion of the effect of shy voters on polling bias because of our study’s limitations—e.g., there is no measure of who is likely to engage in socially desirable responding (Berinsky and Lavine 2012). At the same time, our results also suggest some shy voters exist and that the pressure they feel is a function of the partisan context in which they live, and that their attitudes and preferences are very distinct from otherwise similar non-pressured voters. Whether pressured voters are shy or persuadable must remain an open question, but having identified which voters face which types of pressures gets us one important step closer to predicting their behavior.

Social desirability bias

The Shy Trump Voter Hypothesis is not the first time a polling miss was blamed on voters misrepresenting their voting intentions. In April 1992, pundits who predicted Britain’s first Labour government since 1979 were shocked as the Conservatives easily held onto their majority. Some analysts blamed “Shy Tory Voters” or “Closet Conservatives,” who were ashamed to admit they were voting to keep the status quo in Westminster (Jowell et al. 1993). The theory went that some working-class voters did not want to admit they voted against their class interests in order to avoid a potential tax increase under a Labour government.

As this example makes clear, “shy” voters is the complement to a longstanding, and much studied, phenomenon within polling known as social desirability bias. Applied to turnout, it would be analogous to the reasons surveys underestimate not voting (not the reasons surveys overestimate voting). Clearly, when it comes to turnout, these are two sides of a coin. Surveys overestimate voting because respondents feel pressure to say they engage in a socially desirable political behavior (voting)—a form of polling bias observed all over the world (Blair et al. 2020; Karp and Brockington 2005). But the same sense of civic duty that survey respondents view as pressure to say they voted is also the reason they might be shy about saying they did not vote. The Shy Trump Voter Hypothesis claims that supporting Trump is socially undesirable in the same way not voting is socially undesirable.

Survey respondents reap virtually no rewards from answering questions and so the cost of revealing something that is even a tiny bit embarrassing might lead some to prevaricate (Blair et al. 2020; Zaller and Feldman 1992). For example, Carlson et al. (2020) demonstrate that some experience physical discomfort from even mild political disagreements in discussions—so it is not difficult to imagine that some items would cause discomfort for some respondents in a lengthy survey about politics. Social desirability bias has been shown to bedevil efforts to measure racial and gender attitudes (Krupnikov et al. 2016), immigration attitudes (Janus 2010), and is thought to be the mechanism behind the Bradley effect in voting (when polls systematically overestimate support for black candidates, see Vogel and Ardoin 2008). The pressure to give the socially desirable answer to survey questions may be greater when the survey is conducted by a live interviewer than when it is conducted online, because the respondent wants to please the interviewer (Holbrook et al. 2003).

Given that survey mode is thought to affect bias when it comes to estimating turnout, because respondents feel more social pressure in live-interview situations, evidence that online and live interviews reveal similar support for Trump would suggest social desirability bias is not a major problem when it comes to measuring public support for Trump. In the 238 battleground state surveys AAPOR examined (Kennedy et al. 2018), underestimates of Trump support were actually lowest in the surveys with live interviewers (also see Pew 2017 and Clinton et al. 2021; Clinton et al. 2022). The pattern is problematic for the Shy Trump Voter Hypothesis since the opposite pattern holds when it comes to under-reporting non-voting (Holbrook et al. 2003; Stocke 2007).

And the AAPOR reports are in good scholarly company. A 2017 Pew report also noted similar polling inaccuracies in places where voters were least likely to be shy (e.g., Trump strongholds where it might have been more socially desirable to support Trump). Plus other Republican candidates outperformed polls even though those other candidates were not Trump (Pew 2017; Clinton et al. 2021; Clinton et al. 2022). Shortly before the 2016 election, Alexander Coppock wrote about an experiment he conducted to test for social desirability for the Washington Post’s Monkey Cage feature. Coppock (2016) conducted a list experiment—an experiment designed to circumvent social desirability by allowing respondents to report the number of things they support on a list rather than asking them about each item. Coppock compared the number of items from two lists, one of which asked about voting plans, and found the proportion that planned to vote for Trump in the list experiment was the same as the proportion that said they would vote for Trump when asked directly—again, evidence against the shy Trump hypothesis (later published in Statistics, Politics and Policy, Coppock 2017). Thus, many studies failed to find evidence of the type of bias that the Shy Trump Voter Hypothesis asserts.

Experiments are key for isolating variation in survey response due to social desirability from the constellation of other factors that affect answers to survey questions about vote choice. For example, the non-experimental studies conducted in 1992 cannot determine whether Conservative voters were ashamed of how they were going to vote or simply had a change of heart at the last moment (Jowell et al. 1993). The list experiment described above (Coppock 2016, 2017) removed the social pressure associated with expressions of support for Trump by obscuring Trump support in a list of other items. The 2017 Pew study mentioned earlier randomly assigned different interview modes to survey respondents, ensuring that those with live interviewers were statistically identical to those completing surveys online. In the Pew experiment, social pressure was removed by removing the live interviewer. Still other experiments randomly assigned respondents the direct question about vote choice or a question about which candidate the subjects’ neighbors plan to vote for (Claassen and Ryan 2016). This experiment removed social pressure by allowing respondents to discuss their voting plans under the guise of discussing what their neighbors plan to do. In fact, the Trafalgar Group, one of the rare surveys that predicted a Trump victory in 2016—not just overall, but in Pennsylvania, Michigan, Wisconsin, Florida, and North Carolina—deployed this method to adjust results (Stanton 2020).

Yet another experimental design—the fake-good/fake-bad experiment—doesn’t remove social pressure, but isolates it as a potential source of variation in reported vote choice (see Klar and Krupnikov 2016 for a description of this experiment, though not as applied to the shy voter question). In this experimental setup, subjects are randomly assigned to provide the answer to the vote choice question they believe would make the best impression or the worst impression on others.

Thus, experiments are able to isolate the effects of social pressure. However, these designs tend to provide overall estimates and the preponderance of evidence finds small overall bias due to shy Trump voters. In contrast, when scholars break down the results for different subgroups, many studies reveal evidence that some survey subjects are reluctant to admit to supporting Donald Trump in polls (e.g., Dropp 2016; Enns et al. 2017; Brownback and Novotny 2018; Stout et al. 2021; Wozniak et al. 2019; Klar et al. 2016). These scholarly inquiries have examined the shy Trump voter hypothesis using a variety of different methods. For example, Dropp (2016) compares voting plans across different survey modes (see discussion above) and finds small overall differences. But among more educated respondents, Dropp finds significant differences in Trump support across survey modes (higher support when survey was conducted via the internet rather than by phone). Klar et al. (2016) reason that individuals that tend to self-monitorFootnote 1 are more susceptible to social desirability biases and therefore look for evidence of bias within that subgroup. They find a positive correlation between high scores on a self-monitoring index and support for Trump. Stout et al. (2021) conduct a very similar study in which they find that Trump voters that say they also voted for Obama score high on self-monitoring. They conclude that self-monitoring Trump voters inflate the true number of Obama-Trump cross-overs. Brownback and Novotny (2018) conduct a list experiment to test whether implicit agreement with Trump differs from explicit measures of the same (e.g., the item is in a list with other items and so subjects can indicate agreement with Trump by including it in the count of items with which they agree without explicitly responding to a question about how frequently they agree with Trump). Like the Coppock (2016) list experiment mentioned above, they find small differences overall. But unlike Coppock they find larger differences concentrated within specific subgroups of the electorate (especially Democrats).

The key to reconciling what appears to be a scholarly debate likely lies in recognizing the possibility that significant social desirability bias can exist among individuals AND, if multiple biases counter one another, there might be no evidence of bias in the aggregate. Some studies find evidence that some in the electorate behave as if expressing support for Trump is socially undesirable. Other studies indicate shy Trump voters do not result in large aggregate bias in polls. Are these contradictory? Not necessarily. If social desirability biases associated with candidate support affect both major candidates—not just Trump—then it is possible to have shy individual voters, but small aggregate bias. After all, non-experimental studies like the call back studies mentioned earlier (Kennedy et al. 2016; Pew 2017) show that there are both voters who said “Clinton” in the survey and then said “Trump” when called back and voters who said “Trump” in the survey and “Clinton” when called back. And while comparing experimental groups gives an overall estimate of bias due to social pressure, these overall estimates do not reveal which individuals exhibit Trump, Clinton, or Biden shyness.

Who believes a socially desirable response exists?

Just because many individuals believe that there is a socially desirable response to a question, that does not necessarily mean that surveys will report biased public opinion levels. The level of bias due to social desirability in a survey depends on a number of factors beyond how many people believe there is a socially desirable response. Some of those factors we are not examining in this particular study—e.g., the percentage of respondents who are inclined to answer survey questions in a socially desirable manner. Some we will look at. For example, social desirability bias only exists if people who give the socially desirable answer privately hold a different belief. There is no bias if people’s actual opinion aligns with what they believe the socially acceptable opinion is. We will also determine who believes that there is a socially desirable answer to the vote choice question and whether there is agreement in which response is the socially desirable one.

Beliefs about socially desirable responding are likely responses to perceptions of how an individual believes their friends and family would answer the question. We focus on the social networks in which people are embedded in part because prior research demonstrates, “Interpersonal discussion outweighs the media in affecting the vote” (Beck et al. 2002, p. 57). Of particular interest are voters facing “social cross-pressure”—that is, those embedded in social networks that support a candidate the individual does not—because these individuals might feel pressure to say they support the candidate most of their peers support to remain in good esteem with their peers. As such, this type of social pressure cuts against their partisan predispositions, thus constituting a cross-pressure. We have identified a group of voters we feel are likely to say one thing in a survey and then do another in the voting booth, but we don’t know very much about them yet. Are they socially cross-pressured? Are they misleading pollsters or are they simply persuadable?

Political science as a discipline has long been interested in cross-pressures (a centerpiece of the classic Columbia School research, Lazarsfeld et al. 1944; Berelson et al. 1954). In these studies, the authors focused on individuals whose partisan candidate preference seemed to conflict with the partisan leanings of groups with which the individuals identified. More recently, scholars have studied instances where partisan choice seems to conflict with issue-based choice (e.g., Hillygus and Shields 2008). In any case, cross-pressured individuals appear very distinct, especially when it comes to being persuadable. Hillygus and Shields (2008) highlight this aspect of cross-pressured voters in their title, The Persuadable Voter.

As mentioned above, we define cross-pressure here as conflict that arises between an individual’s partisan candidate preference and the perceived partisan preferences of that individual’s social network. This approach is related to research investigating the consequences of political disagreement in social networks (Huckfeldt et al. 2004; Mutz 2006)—again, work suggesting that these types of individuals are politically distinct. Our work also builds upon several recent investigations that define social cross-pressure in nearly the same way we do. Karpowitz et al. (2011) find that individuals who perceive they support a different candidate than most people around them prize privacy in the voting booth much more than others. Lupton et al. (2015) find that such individuals rely less on their partisanship in forming other attitudes and behaviors. And Brader et al. (2014) find that socially cross-pressured individuals are less engaged in politics across a variety of metrics.

In sum, our analysis of socially desirable responding during the 2020 election will answer a series of questions in the following order. First, we will begin by determining whether individuals believed that there was a socially acceptable way to answer the question about how they were voting. Second, we will examine what types of people in particular believed there was a socially desirable answer and how that answer differs depending on the respondent. Finally, we will run an analysis looking at cross-pressured voters to see how their responses likely differed from voters who did not face social cross-pressures. In the end, the total of these analyses will help us reach a conclusion about the likelihood of bias in polling during the 2020 US presidential election.

Data and methods

There are many methods available for the study of social desirability in survey response. We can divide these methods into three types: (1) identifying questions that are prone to socially desirable responding; (2) identifying survey respondents that are prone to socially desirable responding; and (3) alleviating the consequences of questions or respondents who are prone to socially desirable responding. Examples of third type of method include the list experiment which provides respondents with a means of hiding their true response (see e.g., Glynn 2013; Smallpage et al. 2023) or question wordings that provide excuses for the respondent to give the socially undesirable response (see e.g., Duff et al. 2007; Krupnikov et al. 2016). Examples of the second type, like the Marlowe-Crowne Social Desirability scale (see e.g., Ivarsflaten et al. 2010; Peffley et al. 1997) or the self-monitoring scale (see e.g., Connors et al. 2019; Huang et al. 2023), attempt to measure how much the respondent cares about the judgment of others. The logic is that people are more likely to share opinions that are likely to be unpopular if they do not care about seeking social approval.

The method we are implementing—the best impression/worst impression experiment—is of the first type (for studies using this method see Holbrook et al. 2003; Klar and Krupnikov 2016; Claassen and Ryan 2016; Connors 2023). Respondents are asked to provide the answer that will provide the best or worst impression on others instead of their true response. Because they are not revealing their own opinion but merely stating their beliefs about what other people think, there should be less concern about socially desirable responding. As a result, even survey respondents who are deeply concerned about managing the impression they are giving pollsters should answer these questions truthfully.

While the goal is primarily to identify whether the vote intention question itself is prone to socially desirable responding, we should acknowledge that not all individuals will view social desirability the same way. For example, people might believe that there is an answer that is clearly socially acceptable, but differ about what that answer is. Also, some people may not believe there is a socially desirable answer at all. Some of our analyses will examine differences in how respondents view social desirability when it comes to which candidate they say they support. To be clear, this method does not close the circle on bias by comparing vote intentions to final votes. In fact, some might not revert from socially desirable vote intentions to make different choices in the privacy of the voting booth because they believe there is only one socially acceptable candidate in the election.

We fielded the best impression/worst impression experiment to a national sample administered to about 1800 respondents from October 23rd to 29th, 2020 using Qualtrics Panels. Though not a probability sample, quotas and raking weights ensure age, region, gender, race, and education match the Census estimates of the same. Respondents were randomly assigned to either the worst impression or best impression prompt below:

Please select the answer you believe will make the WORST impression/BEST impression on others—even if this answer does not actually describe your true feelings.

If the 2020 presidential election were being held today, would you vote for Donald Trump, the Republican, or Joe Biden, the Democrat?

  • Donald Trump

  • Joe Biden

In addition, we asked respondents (well before the experiment) about their voting plans (same wording as the last sentence in prompt above)—we call this the direct condition. Finally, in order to identify which individuals exhibit which type of shyness, subjects that were initially in the best impression group were later given the worst impression prompt and vice versa (after answering a variety of other survey questions).

Results

In Fig. 1, we present the proportion of Trump voters in each initial experimental condition—here we report only the first round of the experiment (the best/worst impression condition voters were assigned initially). According to the Cook Political Report, 47% of the national vote went to Donald Trump. In our sample, 47% say they intend to vote for Trump in the direct question and 47% say Trump would make the best impression on others.Footnote 2 Comparing the percent for Trump in the direct condition to the worst impression condition, about 8 percentage points fewer say they will vote for Trump than in the worst impression condition. And the best impression comparison to the worst impression condition is identical (with rounding). The standard error of the direct minus worst comparison is 2.3% and the standard error of the best minus worst comparison is 2.1%. Therefore, both differences are statistically significant at the 95% confidence level.

Fig. 1
figure 1

Voting plans by experimental condition. Note 95% Confidence interval bars

Full size image

So, there was clearly potential for social pressure to matter in the 2020 election. The direction here tells us about the stereotypical direction of social pressure—that many feel saying they would vote for Trump will make a worse impression on others than saying they would vote for Biden. However, this is not a straightforward estimate of how much survey estimates will differ from voting behavior. The problem is that many individuals might feel social pressure, but vote for the person they said they would vote for in the survey nevertheless. In fact, social pressure may reinforce sincere voting preferences for many voters. For example, imagine someone in 2020 who is certain to vote for Biden and believes that expressing support for Trump would lower the esteem of friends, acquaintances, co-workers, and so on. For this person, social pressure increases the likelihood of being consistent in surveys and the voting booth. Hence, the mere existence of social pressure will not necessarily create bias.

Investigating who feels pressure and when pressure might lead to polling errors

This is where our follow-up to each best impression prompt with the worst impression prompt and vice versa comes in. To begin, the modal responses were identical regardless of whether respondents were first asked about best or worst impressions. About 76% named the same candidate in both conditions. Nevertheless, a large group offers different answers depending upon whether they were asked about making the best or worst impression on others: about 24 percent. Theoretically, these voters are revealing important information about perceptions that one candidate is more socially acceptable than the other. To our knowledge, no prior research in which the best/worst impression experiment was fielded followed-up with the opposite condition, and therefore, no prior research has been able to identify individuals expressing this type of social pressure. Because our approach allows us to distinguish pressured individuals from individuals that do not believe others care which candidate they support (specifically, they don’t believe others’ impression of them will be better or worse because of the candidate they support), we are able to identify exactly which respondents offered different answers in different conditions and what pattern those responses followed.

As one might expect, in 2020, the dominant pattern is to say Biden in the best impression condition and then switch to Trump in the worst impression condition. But dominant is not universal. Only 16% of the sample follow this pro-Biden pattern. Although we are not aware of any Shy Biden Voter Hypothesis, fully 8% of the sample say Trump in the best impression condition and then switch to Biden in the worst impression condition—the pro-Trump pattern. And although the difference in these patterns need not resemble the overall difference between the initial best and worst impression treatment (note: both are 8% in the same direction despite the quantities being independent), the symmetry—at a minimum—suggests that this individual-level variation and the overall best/worst group differences share similar structures.

The key becomes figuring out exactly which individuals are likely to respond to this social pressure by saying one thing in a survey and doing another in the privacy of the voting booth. Consider these four types of pressured voters: (1) a Biden voter feeling pressure to support Biden, (2) a Trump voter feeling pressure to support Trump, (3) a Biden voter feeling pressure to support Trump, and (4) a Trump voter feeling pressure to support Biden. Type 1 and type 2 voters seem very unlikely to bias a survey by giving an incorrect answer to the vote choice question because their preference and the pressure they feel are reinforcing. However, type 3 and type 4 voters seem much more likely to say one thing in a survey (because of social pressure), and then do another in the privacy of the voting booth. And we note that although shy Biden voters have received very little attention, an exhaustive analysis of pressured voters would be incomplete without acknowledging the possibility of their existence.

Having elaborated on the four types of pressured voters, our next task is to operationalize each empirically. Ideally, we would observe the actual votes of subjects in order to create the types. This is impractical for legal reasons: whether one voted is a matter of public record, whom one voted for is not. And survey questions about vote choice are also problematic since the goal is to quantify bias in those very questions. The best alternative, in our view, is to use party identification. Although political scientists disagree about many things, they agree party identification tops all other alternatives when it comes to predicting vote choice. Unfortunately, it is difficult to guess which candidate is favored for those identifying as Independents. Therefore, Independents are excluded in our analysis. Doing so divides the pressured portion of the sample into four types:

Type 1: Democrats (by party identification) exhibiting the pro-Biden pattern of social pressure (Biden Best, Trump Worst).

Type 2: Republicans exhibiting the pro-Trump pattern of social pressure.

Type 3: Democrats exhibiting the pro-Trump pattern of social pressure.

Type 4: Republicans exhibiting the pro-Biden pattern of social pressure.

In Fig. 2, we graph the percent of the sample in each of the four types outlined above. We use negative values to provide contrast for valence of pressure. Positive is pressure to say Trump, while negative is pressure to say Biden. The absolute values are the percent of the sample (by partisanship) that feels each type of pressure. The total percent of the sample that named a different candidate in the different conditions is twenty-four, of which about twenty percent were partisans (after excluding the 4% that were Independents). Therefore, the sum of the absolute value of the percentages of the sample in each bar equals about twenty percent.Footnote 3

Fig. 2
figure 2

Percent of sample by type of social pressure. Note We use negative values to provide contrast for valence of pressure. Positive is pressure to say Trump, Negative is pressure to say Biden. The absolute values are the percent of the sample that feels each type of pressure

Full size image

The first and second bars in Fig. 2 do not cancel out, but neither of these groups is likely to say one thing in a survey and do another in the voting booth. In all likelihood, social pressure compels both types of respondents (Type 1 and Type 2, above) to answer vote choice questions sincerely. For these respondents, social pressures do not contradict personal partisan preferences.

The last two bars in Fig. 2, however, represent respondents who may have some incentive to name one candidate in a survey and vote for the other in the voting booth. The pattern of responses to the best/worst condition prompts suggests social pressures might cut in a different direction from personal partisan preferences for these respondents. However, there are equal numbers going in opposite directions: those most likely to be shy Trump voters are canceled out by an equal number of likely shy Biden voters.

Finally, turning to the remaining four percent of pressured voters that are independents (not in the graph), three percent follow the pro-Biden pattern (Biden for best, Trump for worst) and one percent follow the pro-Trump pattern. Would social pressure cut against personal preference for independents? It is difficult to guess, but even if all the pressured independents were shy voters, it would result in a maximum two percent bias (because shyness cuts in both directions for independents too).

The bottom line is that our study reveals that many voters were feeling a lot of social pressure during the 2020 race for president. But analyses of which respondents feel which types of social pressures provide little reason to attribute much, if any, actual bias in the polls due to shy voters.

A new type of cross-pressure: social cross-pressure

Having quantified likely shy voters, we now turn our attention to understanding more about the causes and consequences of these social pressures. More specifically, we are interested in understanding social pressures that might cause individuals to mention a candidate in a survey other than the one for which they eventually vote. The preceding research identifies which individuals we believe are more likely to engage in this behavior (Type 3 and Type 4 above). In the coming section, we will investigate whether the individuals we identified tend to be embedded in social networks that support a candidate that the individual likely does not. We will then compare these socially cross-pressured voters to other voters in more detail.

We have theorized that when voters indicate that different candidates would make different impressions on others (e.g., one would make the best impression and the other would make the worst impression), it means that their family, friends, and other acquaintances tend to favor one candidate. Fortunately, elsewhere in the survey, respondents were asked about their perception of the level of support for each candidate among their friends, in their community, and in their state. Thus, whether the social pressure pattern of best/worst impression responses corresponds to candidate support in the respondents’ social circles is a testable proposition.

In Table 1, we regress each respondent’s best/worst impression pattern on their perceptions about which candidate their friends, others in their community, and others in their state support. The multinomial logit regression provides separate estimates of the effects of these social groups on the pro-Biden pattern (best Biden, worst Trump) and the pro-Trump pattern (best Trump, worst Biden) compared to those mentioning the same candidate in both conditions. The results in the first column of Table 1 indicate that those with more pro-Trump friends were significantly more likely to give the pro-Trump pattern in the best/worst conditions than to mention the same candidate in both conditions. Likewise, those with more pro-Biden friends (the negative coefficient for “Friends Support Trump” in the lower portion of Table 1) were significantly more likely to give the pro-Biden pattern in the best/worst conditions than to mention the same candidate in both conditions. Interestingly, less proximate social circles had negligible effects. These results support the theory that social pressures lead individuals to mention different candidates in the best/worst conditions.

Table 1 Best/worst pattern as a function of trump support in different social circles
Full size table

Critically, the impact of friendship remains robust in the second column of Table 1 despite having added a variety of control variables, including partisanship. Of the control variables, partisanship is the most predictive of whether respondents give the pro-Trump or pro-Biden pattern of answers to the best/worst impression questions. This is consistent with the pattern in Fig. 2 in which most partisans respond in ways that suggest social pressures reinforce party identifications. And the effect of friendship is smaller in the second column because partisanship is also strongly correlated with friendship. In an era of partisan sorting and polarization, that comes as no surprise. However, the independent effect of friendship reveals that some partisans buck the trend. Furthermore, partisans embedded among friends that support the other party’s candidate respond to the best/worst impression questions differently than partisans among likeminded friends.

How many individuals buck the trend of partisan sorting and report friendship networks dominated by people perceived to support the other party’s candidate? To answer this question, we created two dummy variables. One captures individuals in predominantly Trump supporting networks, those that say they think 75% or more of their friends support trump. The other captures individuals in predominantly Biden supporting networks (we only have the one friendship question, so those that say they think 25% or fewer of their friends support Trump are coded as being in Biden dominated networks). Measured in this way, about 14% of Democrats are in pro-Trump networks and about 10% of Republicans are in pro-Biden networks.

Do individuals with friends that support the other party’s candidate look different? Among Republicans, being in a pro-Biden network of friends makes one almost 9% more likely to give the pro-Biden answer pattern (Biden Best Impression, Trump Worst Impression) and about 3% less likely to give the pro-Trump answer pattern—than Republicans with pro-Trump friends. Among Democrats, being in a pro-Trump network of friends makes one about 5% more likely to give the pro-Trump pattern (Trump Best Impression, Biden Worst Impression) and about 18% less likely to give the pro-Biden pattern—than Democrats with pro-Biden friends.Footnote 4 The reason friendship networks matter over and above party ID is (1) some partisans are friends with people from the other party and (2) being in friendship networks dominated by the other party makes Republicans less likely to evidence feeling social pressure to say Trump and more likely to evidence feeling social pressure to say Biden—and vice versa for Democrats.

Clearly partisans with friends that support the other candidate are more likely to respond to the best/worst impression questions in ways that indicate they feel pressure to say they support the other party’s candidate. These response patterns indicate these individuals are feeling cross-pressures. They identify with the party of one candidate, but their answers to the best/worst impression questions indicate that they feel revealing their support for a candidate their friends don’t support would make a bad impression on others.Footnote 5

In order to study the structure of these cross-cutting social pressures further, we isolated partisans giving the best/worst response pattern favoring the other party’s candidate by creating a dummy variable that takes on a value of one for Type 3 and Type 4 voters (see above) and zero for Type 1 and Type 2 voters. The goal is to measure when responses to the best/worst impression questions indicate voters feel cross-pressured relative to their party identification (note, we cannot create this measure for independents and therefore exclude them from this analysis). We model whether individuals appear cross-pressured in this way as a function of partisanship, the measure of which candidate the respondents’ friends supported, and a statistical interaction between these two variables to operationalize the hypothesis that partisanship will function differently depending on whether social pressure from others—friends empirically—is reinforcing or cross-pressuring.

The results in Table 2 demonstrate that the key to determining which partisans indicate feeling social pressure to support the other party’s candidate in the best/worst battery are friendships in which the other party’s candidate is supported. All of the coefficients are robust. The dummy variable for Republican party identification indicates Republicans are more likely to indicate feeling social pressure to support Biden when their friends support Biden (e.g., when “Friends support Trump” equals zero). The coefficient for “Friends support Trump” indicates Democrats (e.g., when Republican equals zero) are more likely to indicate feeling social pressure to support Trump when their friends support Trump. Finally, the robust interaction term indicates the effect of each is different depending on the value of the other. The origin of partisans reporting pressure to support the other party’s candidate in the best/worst battery is indeed social. The model shows that these are Democrats among friends that support Trump and Republicans among friends that support Biden.Footnote 6

Table 2 Cross-pressured partisans as an interactive function of partisanship and trump support among friends
Full size table

Our focus on Type 3 and Type 4 voters was located exclusively within the group of survey respondents that indicated different candidates in the best/worst impression conditions (e.g., compared to Type 1 and Type 2 voters). However, we also note that within the group of respondents that indicated the same candidate in both conditions, a small number of partisans mentioned the other party’s candidate. By far, Trump was the dominant candidate mentioned by Republicans who selected the same candidate in both conditions (ditto for Biden among Democrats who selected the same candidate in both conditions). But a handful of partisans who selected the same candidate in both conditions named the other party’s candidate. Although these individuals named the same candidate twice, and therefore seem to not feel social pressure according to the standard interpretation of the best/worst impression experiment, we note that the two patterns are different and wanted to investigate whether these individuals might be similar to cross-pressured partisans. To assess whether partisans mentioning the other party’s candidate in both conditions are experiencing social cross-pressure, we ran the same regression comparing them to partisans mentioning their party’s candidate in both conditions (in the second column of Table 2) and then combined them with Type 3 and Type 4 voters (in the third column of Table 2). The results indicate nearly identical structure in all three columns. Importantly, these additional voters experiencing social cross-pressure bring the total of such voters to about ten percent of the sample—a much bigger group than the two percent of voters that are Type 3 or Type 4 voters.Footnote 7

Discussion

In our final analysis, we return to the vote intention question to see whether partisans experiencing social cross-pressures say they will vote for the party with which they identify or for the candidate they say would make the best impression on others. Recall, our cross-pressured partisans are individuals whose party identification is different than the party of the candidate they say would make the best impression on others (this applies to both ways of assessing cross-pressures above). As a result, their vote intention must be inconsistent with either their partisanship or the pressure they feel. Which way they go says a lot about potential bias. Cross-pressured partisans that intend to vote for the candidate from the party with which they identify are unlikely to cause bias. They are unlikely to cause bias because their vote intentions reveal a plan to vote for a candidate other than the one they feel pressure to say they support. On the other hand, cross-pressured partisans that intend to vote for the candidate they say would make the best impression on others are in a position to bias the poll because they might revert to the candidate from the party with which they identify in the privacy of the voting booth. Because voting decisions are private, we will never know for certain which route these individuals take. But, just as physicists study potential energy (e.g., a boulder sitting at the top of a steep mountain), our final analysis sheds light on how much potential bias exists.

Our analysis reveals that 67% of cross-pressured partisans say they intend to vote for the candidate from the other party. On one hand, 67% is a long way from 100%. On the other hand, 67% is a massive number compared to the analogous number among partisans that are not cross-pressured. Among partisans (independents are excluded) whose partisanship is consistent with the candidate they say would make the best impression on others, only about 1% say they intend to vote for a party other than the one with which they identify. Returning to the concept of potential energy, in terms of potential bias, our “cross-pressured bias boulder” is over half-way to the top of the mountain.

In terms of survey bias, it could be that these cross-pressured partisans are especially likely to give biased responses in surveys. The bad news is that there are quite a few cross-pressured partisans—about ten percent of the sample (using both measures of cross-pressure above)—and the pattern of their survey responses indicates potential to bias polls. However, their potential to create bias is reduced because only 67% of cross-pressured partisans indicate a vote intention that is inconsistent with their partisanship (67% of 10% is only 6.7%). Moreover the good news, with respect to aggregate bias, is that potential bias runs in both directions and therefore tends toward zero in the aggregate (at least it did in 2020).

We emphasize that we have not demonstrated cross-pressured partisans give biased responses. It is possible that every individual actually voted for the person they said they intended to vote for. We cannot observe actual voting behavior to compare to survey reports. Nevertheless, we have demonstrated that cross-pressured partisans are more likely to say they intend to vote for the other party’s candidate than unpressured partisans. Accordingly, we conclude our analyses with an interesting question about cross-pressured partisans. Are they closet partisans inflating the polling numbers of the other party’s candidate or are they persuadable partisans whose stated vote intensions break with staunchly loyal partisanship?

Conclusion

What does our examination of the Shy Trump Voters Hypothesis allow us to conclude? This experiment was not designed to tabulate exactly how many shy Trump voters there were and, as a result, we cannot make any pronouncements about whether shy Trump voters are driving the recent trend in Democratic polling bias (Clinton et al. 2021; Clinton et al. 2022; Panagopoulos 2021). Our results do suggest that shy Trump voters could exist, but at the same time they suggest the potential for shy Biden voters as well. If both shy Trump and shy Biden voters exist, then that lessens the probability that shy voters cause major problems for aggregate predictions.

Even if we cannot conclusively rule out polling bias due to shy Trump voters, it would probably be useful to eschew use of the Shy Trump Voter Hypothesis in favor of the more general and more useful theory of social desirability bias. Trump is indeed a polarizing and titillating political figure. However, Trump adds nothing unique to the pressures respondents feel when answering survey questions about their voting plans. Social desirability pressures may rise and fall across time and candidates, but the way these pressures function remains the same, even if the size of the effects vary.

Accordingly, we note that we found evidence of social desirability pressure among both Trump supporters and Biden supporters. Further analysis revealed that the social context in which respondents were embedded was highly predictive of which respondents seem to feel social desirability pressure. And again the symmetry is striking—it really isn’t about Trump per se. Democrats surrounded by Trump supporters and Republicans surrounded by Biden supporters were all more likely to feel social desirability pressures when asked about their voting plans. Finally, and perhaps most importantly, these socially cross-pressured partisans (e.g., Democrats surrounded by Trump supporters and Republicans surrounded by Biden supporters) are associated with highly unusual voting plans. Voters reporting these types of social cross-pressures are far more likely to say in surveys that they plan to vote for a candidate other than their own party’s candidate. On one hand, we show these individuals have the potential to return to their partisan predisposition in the privacy of the voting booth and bias polls in the process. But it is also possible that cross-pressured individuals are simply more persuadable than unpressured partisans. After all, a campaign consultant would be hard pressed to identify a group in which more people indicate an intention to vote against the party with which they identify. Democrats surrounded by Republicans who say they intend to vote Republican and vice versa would surely merit significant attention from both Democratic and Republican campaigns.

To conclude, we confess that we leave open the question of what socially cross-pressured survey respondents actually do. Are they misleading pollsters or are they revealing an intent to break with their party in the voting booth? It is likely that both happen over the course of a campaign. For example, in explaining why the Labour Party closed a considerable gap in the polls in the final weeks prior to the 2017 UK General Election, Mellon et al. (2018) argue that Jeremy Corbyn was able persuade former Labour voters who were undecided as to how they were going to vote. But it is also possible that there were shy Labour voters prior to the active campaign who believed that their social circles would not find a Corbyn-led Labour Party acceptable, only to find during the campaign that they had an incorrect perception of their friends’ beliefs allowing them to be open about their support. From the perspective of the campaign professional, there is a world of difference between these two possibilities. “Shy” voters are problematic and make it difficult to trust polls, but persuadable voters are the ones campaigns want to heap attention upon. And we hasten to add that this seems to be an open question more broadly. Monson et al. (2011) found socially cross-pressured individuals prize privacy when voting. Perhaps they prize privacy because they plan to vote counter to the social pressures they feel—suggesting they were misleading pollsters. But work on cross-pressures in general treats cross-pressured voters as persuadable (see the classic Columbia studies; Hillygus and Shields 2008).

If socially cross-pressured respondents are to be believed, then Democratic campaigns should focus on Republicans in close states who say they are surrounded by Democrats and vice versa. And since campaigns are increasingly using commercial data, perhaps this suggests they should find the pickup truck drivers that shop at Target and the Prius drivers that shop at Walmart (Hetherington and Weiler 2018; i.e., find commercial behaviors that are associated with having opposite party friends). In any case, campaigns will not find a group of individuals more likely to say they plan to vote for the other party’s candidate than the partisans socially embedded among opposing partisans.