Abstract
The reluctance of people to get vaccinated represents a fundamental challenge to containing the spread of deadly infectious diseases1,2, including COVID-19. Identifying misperceptions that can fuel vaccine hesitancy and creating effective communication strategies to overcome them are a global public health priority3,4,5. Medical doctors are a trusted source of advice about vaccinations6, but media reports may create an inaccurate impression that vaccine controversy is prevalent among doctors, even when a broad consensus exists7,8. Here we show that public misperceptions about the views of doctors on the COVID-19 vaccines are widespread, and correcting them increases vaccine uptake. We implement a survey among 9,650 doctors in the Czech Republic and find that 90% of doctors trust the vaccines. Next, we show that 90% of respondents in a nationally representative sample (n = 2,101) underestimate doctors’ trust; the most common belief is that only 50% of doctors trust the vaccines. Finally, we integrate randomized provision of information about the true views held by doctors into a longitudinal data collection that regularly monitors vaccination status over 9 months. The treatment recalibrates beliefs and leads to a persistent increase in vaccine uptake. The approach demonstrated in this paper shows how the engagement of professional medical associations, with their unparalleled capacity to elicit individual views of doctors on a large scale, can help to create a cheap, scalable intervention that has lasting positive impacts on health behaviour.
Similar content being viewed by others
Main
COVID-19 is a salient example of a disease with profound economic, social and health impacts, which can be controlled by large-scale vaccination if enough people choose to be vaccinated. Nevertheless, a large percentage of people are hesitant to get a vaccine, preventing many countries from reaching the threshold necessary to achieve herd immunity9,10. Consequently, rigorous evidence on scalable approaches that can help to overcome people’s hesitancy to take a COVID-19 vaccine is a global policy priority3,4,5. Existing research has made important progress in documenting the roles of providing financial incentives11,12, reminders4,5, information about the efficacy of the vaccines13,14, the role of misinformation15 on the intentions of the public to get vaccinated and, more recently, also on their actual decisions to get a vaccine5 shortly after an intervention. However, little is known about whether cheap, scalable strategies with the potential to cause lasting increases in people’s vaccination demand and uptake exist. A focus on the persistence of the impacts of interventions is especially important for vaccines such as those against COVID-19, which are often distributed in phases to different demographic groups due to capacity constraints, and multiple doses spaced over time are required to avoid declines in protection.
In many surveys across the globe, people report that they strongly trust the views of doctors6. This makes it crucial to understand how people perceive doctors’ views about the COVID-19 vaccine. In this paper, we pursue the hypothesis that reluctance to adopt the vaccine originates, in part, in misperceptions about the distribution of aggregate views of the medical community: many people may fail to recognize that there is a broad consensus in favour of the vaccine among doctors. Furthermore, we argue and show that professional associations can serve as aggregators of individual views in a medical community, by helping to implement surveys eliciting the views of doctors on a large scale. Disseminating information of a broad consensus, when one exists, can lead to people updating their perceptions of doctors’ views and, in turn, may induce lasting changes in vaccination demand and uptake.
Our focus on public misperceptions of the views of doctors is motivated by a widespread concern that media coverage can create uncertainty and polarization in how people perceive expert views, even when a broad consensus actually exists. In terms of traditional media, a desire to appear neutral often motivates journalists to provide a ‘balanced’ view by giving roughly equal time to both sides of an argument7,16, creating an impression of controversy and uncertainty8. Such ‘falsely balanced’ reporting has been shown to be a characteristic element of policy debates ranging from climate change7,16 to health issues, including links between tobacco and cancer, and potential side effects of vaccines8,17. In the context of the COVID-19 vaccines, casual observation suggests that media outlets often feature expert opinions that highlight the efficacy of approved COVID-19 vaccines together with skeptical experts who voice concerns about rapid vaccine development and untested side effects. The media usually do not specify which claims are supported by the wider medical community, leading the World Health Organization to warn media outlets against engaging in false-balance reporting18. Furthermore, polarization of beliefs can arise due to echo chambers—people choosing to be exposed to expert opinions or opinion programmes that fuel their fears of the vaccine or, alternatively, to those who strongly approve of it19,20,21.
We study these issues in the Czech Republic, which is a suitable setting, given the observed level of vaccine hesitancy among a large share of its population, similar to the situation in many other countries. At the time of data collection, the acceptance rate of the vaccine in the Czech Republic was around 65%, compared to 55–90% in other countries globally. At the same time, the Czech Republic ranks close to the median level of trust and satisfaction with medical doctors, based on a comparison of 29 countries6. We provide more background in Section 3.1 of the Supplementary Information.
We start by documenting and quantifying public misperceptions about the views of doctors on the COVID-19 vaccines. Shortly before the COVID-19 vaccine rollout began, we implemented a short online survey among 9,650 doctors. We found strong evidence of consensus: 90% of doctors intend to get vaccinated themselves and 89% trust the approved vaccines. At the same time, we found evidence of systemic and widespread misperceptions of the views held by the medical community among a nationally representative sample of the adult population (n = 2,101): more than 90% of people underestimate doctors’ trust in the vaccines and their vaccination intentions, with most people believing that only 50% of doctors trust the vaccines and intend to be vaccinated.
These findings set the stage for our main experiment, in which we tested whether randomized provision of information about the actual views of doctors can recalibrate public beliefs and, more importantly, cause a lasting increase in vaccination uptake. The experimental design aimed to make progress on two important empirical challenges that are common in experiments on the determinants of demand for COVID-19 vaccines. First, as an intention–behaviour gap has been documented in the context of flu vaccines and other health behaviours22, measuring both vaccination intentions and actual vaccination uptake allows us to test whether treatment effects on vaccination intentions translate into behavioural changes of a similar magnitude. The initial set of studies on COVID-19 vaccination, typically implemented before the vaccines became available, only tested impacts on intentions11,14,15, although recent exceptions exist5,23.
Second, most experiments designed to correct misperceptions about the views of others, and other information provision experiments in various domains, including migration, health and political behaviour, document treatment effects to be substantially smaller when measured with a delay24,25. In theory, the worry is that individual perceptions about the views of doctors might shift between the time when the treatment takes place and when people decide whether to actually get vaccinated, for reasons including regression of perceptions to the mean, biased recall or motivated memory26. Conversely, researchers have suggested that providing facts about a widely shared consensus of trustworthy experts might be resilient to these forces17, as the treatment may reduce incentives to seek new information, and condenses complex information into a simple fact (‘90% of doctors trust the approved vaccines’), which is easy to remember. Understanding whether providing information about medical consensus has temporary or lasting effects on vaccination demand is informative for policy, in terms of whether a one-off information campaign is sufficient, or whether the timing of messages needs to be tailored for different groups of people who become eligible for a vaccine at different points in time, and also whether such an information campaign needs to be repeated in cases of multiple-dose vaccines.
To address these issues, our experiment is integrated into longitudinal data collection with low attrition rates. The treatment was implemented in March 2021. We used data from 12 consecutive survey waves collected from March to November 2021, covering the early period when the vaccine was scarce, later when it gradually became available to more demographic groups, and finally for several months when it was easily available to all adults. This is reflected in the vaccination rates, which increased in our sample from 9% in March to 20% in May and to nearly 70% in July. Then, it grew slowly to 77% at the end of November. This longitudinal, data-collection-intensive approach allows us to estimate: (1) whether disseminating information on the consensus view of the medical community has immediate effects on people’s beliefs and their intentions to get the vaccination shortly after the intervention; (2) whether the effects translate into actually getting vaccinated, even though most of the participants became eligible for the vaccine only many weeks after the intervention; and (3) whether the effects on vaccine uptake are persistent or whether the vaccination rate of untreated individuals eventually catches up, perhaps due to ongoing governmental campaigns, stricter restrictions for individuals who are not vaccinated, or greater potential life disruptions during severe epidemiological periods.
Consensus of the medical community
We conducted a supplementary survey to gather the views of doctors on COVID-19 vaccines in February 2021. The survey was implemented in partnership with the Czech Medical Chamber (CMC), whose contact list includes the whole population of doctors in the country, because membership is compulsory. All doctors who communicate with the CMC electronically (70%) were asked to participate and 9,650 (24% of those contacted) answered the survey. Supplementary Table 1 provides summary statistics and documents that the sample is quite similar, in terms of age, gender, seniority and location, to the overall population of medical doctors in the Czech Republic.
Figure 1 shows the distribution of doctors’ responses. A clear picture arises, suggesting that a broad consensus on COVID-19 vaccines exists in the medical community: 89% trust the vaccine (9% do not know and 2% do not trust it), 90% intend to get vaccinated (6% do not know and 4% do not plan to get vaccinated) and 95% plan to recommend that their patients take a vaccine (5% do not). These responses are broadly similar across gender, age, years of medical practice and size of the locality in which the doctors live: for all sub-groups, we found the share of positive answers to all questions ranges between 85% and 100% (Supplementary Table 2). Using probability weights based on observable characteristics of the entire population of doctors in the country makes very little difference in the estimated distribution of opinions in our survey. Reassuringly, the opinions in our survey are in line with high actual vaccination rates (88%) observed among Czech doctors when vaccines became available27, despite vaccination not being compulsory for any profession, including for doctors.
Supplementary study among the members of the CMC (n = 9,650). a, Distribution of responses to the question “Will you personally be interested in getting vaccinated, voluntarily and free of charge, with an approved vaccine against COVID-19?”. Among participants who answered yes, the dark blue refers to those who reported already being vaccinated, whereas the light blue refers to those who plan to get vaccinated. b, Responses to the question “Do you trust COVID-19 vaccines that have been approved by the European Medicines Agency (EMA) approval process?”. c, Responses to the question “Will you recommend COVID-19 vaccination to your healthy patients to whom you would recommend other commonly used vaccines?” Among participants who answered yes, the dark blue refers to those who would recommend the vaccines even without being asked, whereas the light blue refers to those who would recommend only when asked. In Supplementary Table 2, we show that the distribution of views is similar across various demographic groups and level of seniority.
Longitudinal experiment
Our main sample consists of participants in the longitudinal online data collection ‘Life during the pandemic’, organized by the authors in cooperation with PAQ Research; the data were collected by the NMS survey agency (Methods and Supplementary Methods). The information intervention was implemented on 15 March 2021 (wave 0). We used data from 12 consecutive waves of data collection regularly conducted from March to November 2021. This time span covers the period when the vaccination was gradually rolled out and eligibility rules changed regularly, making the vaccine available for more demographic groups (until June 2021), and a period when vaccination was freely available for the entire adult population (from July 2021).
The sample from wave 0 is our ‘base sample’ (n = 2,101). By design, the sample is broadly representative of the adult Czech population in terms of a host of observable characteristics (for summary statistics, see Extended Data Table 1). In addition, the vaccination rate reported in our sample closely mimics the levels and dynamics of the overall adult vaccination rate in the country (Extended Data Fig. 1). This comparison suggests that attitudes to vaccination in our sample are likely to be representative of the larger population, in contrast to surveys based on convenience samples28. Although this pattern is reassuring, we cannot test and fully rule out a possibility that our sample might not be representative in terms of unobservable characteristics affecting receptivity to the information treatment studied. Furthermore, the response rate in the follow-up waves is high, ranging between 76% and 92%. A large portion of participants (n = 1,212; the ‘fixed sample’) took part in all 12 waves of data collection.
The participants were randomly allocated to either the Consensus condition (n = 1,050) or Control condition (n = 1,051) in wave 0. In the Consensus condition, they were provided with a summary of the survey among medical doctors, including three charts that displayed the distribution of doctors’ responses regarding their trust in the vaccines, willingness to get vaccinated themselves and intentions to recommend the vaccine to patients. In the Control condition, the participants did not receive any information about the survey of medical doctors and only filled the regular part of the longitudinal survey.
In all 12 waves, we asked whether respondents got vaccinated against COVID-19. The main outcome variable ‘vaccinated’ is equal to one if the respondent reported having obtained at least one dose of a vaccine against COVID-19. We also elicited prior beliefs on the views of doctors about the vaccines in wave 0 shortly before the information intervention, and posterior beliefs in wave 1 2 weeks afterwards.
Extended Data Table 1 and Supplementary Table 3 show no systematic differences in the set of baseline characteristics pre-registered as control variables. Nevertheless, because the randomization was not stratified on baseline covariates, there are random imbalances in some covariates, as expected. Some of the larger differences are for variables not included in the set of pre-registered control variables. Specifically, before the intervention, compared to participants in the Control condition, the individuals in the Consensus condition were slightly less likely to be vaccinated themselves (standardized mean difference (SMD) = 0.069), and expected a smaller percentage of doctors to trust the vaccine (SMD = 0.072) or to intend to get vaccinated (SMD = 0.090). As these three variables are highly predictive of vaccination uptake, we report two main regression specifications: (1) with the pre-registered set of control variables, and (2) with control variables selected by the LASSO procedure29. To document robustness, we also report estimates with no control variables and with alternative sets of control variables.
Misperceptions about doctors’ views
To quantify misperceptions about the views of doctors on COVID-19 vaccines, we compared the prior beliefs of participants about doctors’ views, measured before the intervention, with the actual views of the doctors from the CMC survey. We found strong evidence of misperceptions. The average, median and modal guesses are that 57%, 60% and 50% of doctors, respectively, want to be vaccinated (Fig. 2a), whereas in reality 90% of doctors do. The average, median and modal guesses about the percentage of doctors who trust the vaccines are 61%, 62% and 50%, respectively (Fig. 2b), whereas in practice 89% of doctors report trusting the vaccines. A vast majority of participants underestimate the percentage of doctors who want to be vaccinated (90%) and those who trust the vaccines (88%).
A sample of the adult Czech population (n = 2,101). a, Distribution of the prior beliefs of respondents about what percentage of doctors would like to get vaccinated. b, Distribution of the beliefs of respondents about what percentage of doctors trust approved COVID-19 vaccines. The dashed line shows the true value, based on the responses of doctors in the Supplementary study. The red and blue colours show the percentage of those who underestimate and overestimate, respectively, doctors’ own vaccination intentions (a) and trust in the COVID-19 vaccines (b).
The distribution of beliefs reveals that the large underestimation does not originate in two distinct groups of participants holding opposite views of the medical consensus—one group thinking that most doctors have positive views about the vaccines and the other group thinking that most doctors are skeptical about them. Instead, most people expect a wide diversity of attitudes across individual doctors. Of participants, 81% believe that the percentage of doctors who want to be vaccinated is between 20% and 80%. For beliefs about doctors’ trust in the vaccines, this number is 76%. Furthermore, these misperceptions are widespread across all demographic groups based on age, gender, education, income and geographical regions (Supplementary Table 4).
We found several intuitive descriptive patterns that increase confidence in our measures of beliefs. First, beliefs about the vaccination intentions of doctors and their trust in the vaccines are strongly positively correlated (r(2,099) = 0.60, P < 0.001). Second, beliefs about doctor’s trust and vaccination intentions are highly predictive of respondents’ own intentions and uptake (Supplementary Table 4). In the next sub-section, we explore whether this relationship is causal. Third, in Supplementary Fig. 1, we show that misperceptions about the doctor’s views are unlikely to arise due to the inattention of participants to the questions. The results are very similar when we excluded the 4% of participants who did not pass all of the attention checks embedded in the survey, and when we excluded the 10% of participants with the shortest response times.
Intervention impacts on vaccination
We first established the effects of the intervention on posterior beliefs about the views and vaccination intentions of doctors shortly after the intervention. We found that the information provided shifts expectations about the views of doctors (Fig. 3a and Supplementary Table 5). Two weeks after the intervention (in wave 1), the Consensus condition increased beliefs about the share of doctors who trust the vaccines by 5 percentage points (p.p.) (P < 0.001) and beliefs about the share of doctors who want to get vaccinated by 6 p.p. (P < 0.001). Next, the Consensus condition increased the prevalence of people intending to get vaccinated by around 3 p.p. (P = 0.039; Fig. 3b and Supplementary Table 6). When we restricted the sample to those who participated in all waves, we found the point estimate to be slightly larger (5 p.p., P = 0.001).
A sample of the adult Czech population. a, Estimated effects of the Consensus condition on beliefs about the percentage of medical doctors who plan to get vaccinated (left panel) and on beliefs about the percentage of doctors who trust approved COVID-19 vaccines (right panel), measured in wave 1 (29 March; Consensus condition n = 970; Control n = 970). b, The dependent variable is an indicator for an intention to be vaccinated with a vaccine against COVID-19, measured in wave 0 (15 March; Consensus condition n = 1,050; Control n = 1,051) and wave 1 (29 March; Consensus condition n = 970; Control n = 970). We report the results of two specifications: (1) a linear probability regression controlling for pre-registered covariates: gender, age category (6 categories), household size, number of children, region (14 regions), town size (7 categories), education (4 categories), economic status (7 categories), household income (11 categories) and baseline vaccination intentions, and (2) a double-selection LASSO linear regression selecting from a wider set of controls in Extended Data Table 1, including prior vaccine uptake and beliefs about the views of doctors. Markers show the estimated effects and the whiskers denote the 95% confidence interval based on Huber–White robust standard errors. The estimated effects and Student's t-test (two-sided) P values are reported in the figure. No adjustments were made for multiple comparisons. We report estimates for (1) all observations, full sample (diamond and square), and (2) for a sub-sample of participants who took part in all 12 waves (Consensus condition n = 614; Control n = 598), fixed sample (triangle and circle). In the lower part of the figure, we report the timing, the total number of observations and the Control mean for each wave. See Supplementary Section 3.5 for further specification details. Supplementary Tables 5 and 6 show the regression results for a and b in detail, respectively.
Next, we found a systematic, robust and lasting treatment effect on vaccine uptake. Four months after the intervention, when vaccines became available to all adults, we found that participants in the Consensus condition were around 4 p.p. more likely to be vaccinated than those in the Control condition (Figs. 4 and 5). As expected, owing to the gradual rollout of the vaccine during the March to June period, the effect emerged gradually (Extended Data Table 2 provides more information about changes in vaccine eligibility rules). The difference in the uptake rates between the Consensus and Control conditions steadily increased to 4–5 p.p. in July and remained relatively stable thereafter (Fig. 4 and Extended Data Table 3).
A sample of the adult Czech population. Estimated effects of the Consensus condition by survey wave on getting at least one dose of a vaccine against COVID-19. We report the same four specifications as in Fig. 3 (linear probability model with pre-registered controls using full (diamond) and fixed (triangle) samples, and double-selection LASSO linear regression selecting from controls in Extended Data Table 1 using full (square) and fixed (circle) samples). Markers show the estimated effects and the whiskers denote the 95% confidence interval based on Huber–White robust standard errors. The estimated effects and Student's t-test (two-sided) P values are reported in the figure. No adjustments were made for multiple comparisons. We report estimates for (1) all observations, full sample (diamond and square), and (2) for a sub-sample of participants who took part in all 12 waves, fixed sample (triangle and circle). In the lower part of the figure, we report the timing, the total number of observations and the Control mean for each wave. Full sample: Consensus condition n = 807–970, Control n = 800–973; see Extended Data Table 2 for exact n per wave. Fixed sample: Consensus condition n = 614; Control n = 598. Extended Data Table 3 shows the regression results in detail.
A sample of the adult Czech population. This specification chart plots the estimated effects of Consensus on the likelihood of vaccine uptake for a pooled sample across waves 6–11 (when the vaccine was available for all adults). All specifications include wave fixed effects. Markers show the estimated effects, the darker or lighter whiskers denote the 90% or 95% confidence interval, respectively, based on standard errors clustered at the respondent level. No adjustments were made for multiple comparisons. We report a range of linear probability model specifications by sequentially adding sets of control variables in Extended Data Table 1. The main specifications are marked by blue diamonds. We report all specifications for both the full sample (left-hand side) and the fixed sample (right-hand side). Full sample: Consensus condition n = 5,145 (981 clusters = respondents); Control n = 5,137 (983 clusters = respondents). Fixed sample: Consensus n = 3,684 (614 clusters = respondents); Control n = 3,588 (598 clusters = respondents). Extended Data Table 4 shows the regression results in detail.
In Fig. 5 and Extended Data Table 4, we report results from pooled regressions to utilize data from all six waves implemented in July to November, include wave fixed effects and cluster standard errors at the individual level. The estimated treatment effect is significant for both main specifications—when we control for a set of variables selected by the LASSO procedure (P = 0.005) and when we control for the pre-registered set of variables (P = 0.026). The effect is similar when estimated in each of these waves separately (Fig. 4).
The estimated effect size is slightly larger (4.4 p.p.) when we used the specification with LASSO-selected control variables than when we used the specification with pre-registered control variables (3.5 p.p.). Figure 5 shows that this is because the LASSO procedure selects baseline beliefs and vaccination status as relevant control variables, whereas these variables are not included in the pre-registered set. Consequently, both approaches document robust positive treatment effect between 3.5 and 4.4 p.p. Readers who believe that researchers should control for random imbalances in important baseline variables may favour the upper bound, whereas readers concerned about departures from pre-registered analyses may favour the lower bound.
Our finding of a positive treatment effect does not rely on a specific choice of control variables or estimation strategy. First, the effect is very similar when we controlled for various sets of baseline variables other than the pre-registered and LASSO-selected sets, as well as when we controlled for none (Fig. 5 and Extended Data Table 4). Second, the effect is significant at conventional levels when we calculated P values using the randomization inference method (Extended Data Tables 3 and 5). Third, the estimated treatment effect is 5.4 p.p. (P = 0.008) when we used baseline data about vaccination rates, and used a difference-in-difference estimation (Supplementary Table 7). Furthermore, the results are robust to excluding participants who arguably paid less attention (Extended Data Table 5). As in the analysis of vaccination intentions, the estimated effects on uptake are slightly larger when we restricted the analysis to those who participated in all 12 waves.
Differential attrition cannot explain our findings. First, we found that the participation rate is relatively high and does not differ across the Consensus and Control conditions on average. There is also no evidence of differential attrition by baseline covariates, suggesting that different types of individuals were not participating in the Consensus and Control conditions (Supplementary Table 8). We found this pattern for participation in each of the 11 follow-up waves separately as well as when we focused on participation in all waves (being in the fixed sample). As a sensitivity test, we imputed missing vaccination status for those who did not participate in some of the waves and assumed either that (1) their vaccination status has not changed since the last wave for which the data are available, or that (2) their status is the same as the one reported in the earliest next wave for which the data are available. The first approach allowed us to impute all the missing information because we know the vaccination status of each participant in the initial wave. The second approach allowed us to impute the missing information, except in cases when a respondent did not participate in the last wave. The effects are robust (Extended Data Table 5).
The effect of the Consensus condition on uptake is lasting. First, although in the main estimates we focused on the likelihood of respondents getting at least one vaccine dose, a qualitatively similar and significant effect emerges when we focused on the likelihood of participants getting two doses (Extended Data Fig. 2). Second, the treatment effect emerges during a 3-month period, due to availability restrictions, and then is stable across all six follow-up waves covering the July to November period (Fig. 4). Thus, the main effect is not driven by differences in the timing of getting vaccinated. Last, in the September and November waves, we asked about the intentions of participants to get a booster dose. The estimated effect is very similar in magnitude as the effect on uptake of the first dose (around 4 p.p.), suggesting that the information intervention elevates vaccination demand even 9 months after it was implemented (Extended Data Fig. 2).
Documenting such persistence has interesting implications. As the demand for vaccination in the Control condition does not catch up with the Consensus condition over such a long period, the results suggest that the type of vaccine hesitancy reduced by the Consensus condition is resilient to policies, campaigns or any life disruptions that participants were exposed to during the period studied. This includes a severe COVID-19 wave that took place in November 2021 in the Czech Republic, which resulted in one of the highest national mortality rates in global comparisons (see Section 3.1 of the Supplementary Information and Extended Data Fig. 3).
The point estimates of around 4 p.p. imply a relatively large effect size, especially in light of the low costs of the intervention. As the vaccination rate in the Control condition was 70–75% during the July to November period, the Consensus condition reduces the number of those who are not vaccinated by 13–16%. To compare, providing truthful information about the vaccination intentions of other people was shown to increase intentions to get vaccinated by 1.9 p.p.30. Nudging health workers to get vaccinated by referring to vaccinated colleagues has been shown to increase the likelihood of their registering for vaccination by around 3 p.p.31. More generally, the most successful, low-cost behavioural nudges with documented effect on uptake have estimated effect sizes up to 5 p.p.4,5, which is quite similar to the effect of providing information about consensus in doctors’ opinions studied here. In addition, a noteworthy aspect of our study is the documented persistence of the effects, which is another crucial margin for assessing the intervention effectiveness.
The Supplementary Information describes exploratory analyses of how the treatment effect differs across different sub-samples of respondents (Supplementary Table 5 and Extended Data Table 5). Reassuringly, we found that the positive effect on vaccine uptake is concentrated among those who underestimated doctors’ trust and vaccination intentions, whereas no systematic effect was observed among overestimators. In addition, the effect is driven by those who initially did not intend to get vaccinated, in line with the interpretation that the intervention changed the views of individuals who were initially skeptical about the vaccine. Nevertheless, the analysis of heterogenous effects should be treated as tentative because the differences in coefficients are not always significant and we did not adjust for testing of multiple hypotheses.
Given that vaccination status is self-reported, we provide several tests documenting that the observed effect does not arise due to priming or the experimenter demand motivating some people in the Consensus condition to report being vaccinated even when they were not. We begin by noting that the observed treatment effect is lasting and emerged only gradually over several months, as more people became eligible to get vaccinated. By contrast, priming and experimenter demand effects are typically thought to be relevant mainly for responses shortly after a treatment25,32.
To probe more directly, we used two distinct approaches to verify the reported vaccination status in the main dataset. First, inspired by existing work25,33, we used additional data about vaccination status collected for us by a third, independent party among the same sample. As the survey agency, graphical interface and topic of the survey were different from our main data collection, the experimenter demand effect that might be potentially associated with treatment in our main survey is unlikely to affect responses in the third-party verification survey. Only two respondents (one in the Consensus condition and one in the Control condition) reported being vaccinated in the main survey, but reported the opposite in the verification survey (Extended Data Table 6), so mismatch in reporting of being vaccinated is very rare in general and not related to treatment. We arrive at a similar conclusion using the second verification approach that links reported vaccination status with an official proof of vaccination: an EU Digital COVID certificate issued by the Czech Ministry of Health. We showed that respondents in the Consensus condition compared to the Control condition are not less willing or able to provide verifiable information from the certificate (Extended Data Table 6). Finally, we showed that the effect of the Consensus condition on lower prevalence of those reporting not being vaccinated in the main survey is almost fully explained by greater prevalence of those reporting being vaccinated and having their vaccination status verified (Supplementary Table 9). More details about the methods and results of both verifications appear in the Methods section and in Section 3.4 of the Supplementary Information.
Discussion
Our results shed light on the role that misperceptions of the distribution of expert views have in vaccine hesitancy, and also show how this barrier can be lifted by providing accurate information. We provide evidence that (1) the vast majority of medical doctors in the Czech Republic trust the approved COVID-19 vaccines, (2) the vast majority of respondents in a nationally representative survey substantially underestimate the percentage of doctors with positive views of the vaccine, and (3) correcting these misperceptions has lasting positive effects on vaccine uptake. Although existing experiments have made progress in identifying low-cost strategies to increase vaccination intentions4,13,14,15 and uptake5 measured shortly after the intervention, this paper integrates the experiment in longitudinal online data collection and contributes by identifying a low-cost, scalable treatment that has lasting effects on behaviour.
Scientists, and the medical community as a whole, have invested enormous efforts to develop and deliver COVID-19 vaccines. However, much less collective effort has been directed at informing the public of the high levels of trust in the vaccine across the broad medical community. Here we show that professional medical associations can serve as aggregators of individual doctors’ views, by facilitating opinion polls among doctors. Resulting data can be used in campaigns to tackle vaccine hesitancy and also as input for media reports. Although we cannot empirically pin down the sources of the misperceptions observed in our study, we suspect that they originate, at least in part, in a journalistic norm in which balance is often considered a mark of objective and impartial reporting, and a way to attract the attention of news consumers34. Our results strengthen the case for supplementing contrasting views on controversial issues with information about how prevalent such views are35.
To guide efforts to scale up this intervention, we discuss what types of factors may affect its efficiency and how we view the boundary conditions in terms of the applicability of the intervention beyond the context that we studied. We estimate the effects of a one-time intervention, among a sample in which most people probably paid attention to the information. Understanding whether the efficiency of the intervention can be fostered by repeated provision of information, as some research has suggested36, and which modes of delivery, such as media advertisements, text messages or informational mail flyers, can best attract a sufficient degree of attention is an important next step for future research. Next, in many settings, implementing such information campaigns by governments, health insurance companies or healthcare providers may help to facilitate access to the contacts of large numbers of individuals4,5 and to address the need for a trusted source to provide the information intervention. Furthermore, in theory, this type of intervention should have larger effects: (1) the greater the trust in medical doctors in a given country is, and (2) the greater the prevalence of misperceptions about the views of doctors towards a vaccine is. We studied this intervention in a country with an approximately median level of trust in doctors6, which provides some confidence that our findings from the Czech Republic may extend to other settings. At the same time, to our knowledge, because this is the first paper to provide direct evidence of the prevalence and size of misperceptions about the views of doctors on COVID-19 vaccines, we can only speculate how widespread such misperceptions are in other settings. Given that the likely sources of the misperceptions—false-balance reporting and echo chambers—are not specific to the Czech Republic, and given that misperceptions about scientific consensus have been documented in other countries in other domains, including health and climate change24,37, we suspect that this bias in beliefs about COVID-19 vaccines is relatively widespread. We hope to see more research on this front.
Methods
Supplementary survey among doctors
To gather the views of doctors on COVID-19 vaccines, we implemented a survey in partnership with the CMC, to maximize coverage of the medical community. The survey was implemented online in February 2021. Because membership in the CMC is compulsory, the CMC has a list of contacts for the whole population of doctors in the country. The CMC approached all doctors who communicate with the CMC electronically (70%) and asked them to participate in a short survey, using the Qualtrics platform. Of doctors contacted, 9,650 (24%) answered the survey. The doctors in our sample work in all regions of the country, are on average 52 years of age, 64% are female individuals and 62% have more than 20 years of experience. A comparison of characteristics of doctors in our sample and of all doctors in the Czech Republic is presented in Supplementary Table 1.
Main experiment
Sample
Our main sample consisted of 2,101 participants of the longitudinal online data collection ‘Life during the pandemic’, organized by the authors in cooperation with PAQ Research and the NMS survey agency. In March 2020, the panel began to provide real-time data on developments in economic, health and social conditions during the COVID-19 pandemic. We used data from 12 consecutive waves of data collection conducted at 3–4-week intervals between mid-March and the end of November 2021.
The information intervention was implemented on 15 March 2021, which we labelled as wave 0. The sample from wave 0 is the ‘base sample’ (n = 2,101, 1,052 female participants and 1,049 male participants, mean age of 52.9 years (s.d. = 15.98), youngest 18 years of age, oldest 92 years of age). The base sample is broadly representative of the adult Czech population in terms of sex, age, education, region, municipality size, employment status before the COVID-19 pandemic, age × sex, and age × education. Prague and municipalities with more than 50,000 inhabitants are oversampled (boost 200%). Sample statistics are presented in Extended Data Table 1. The sample is close to being representative of the adult Czech population in terms of attitudes to COVID-19 vaccines. The development of the proportion of people getting vaccinated in the Control condition very closely mimics the actual vaccination rates in the Czech Republic (Extended Data Fig. 1), when we weighted the observations in our sample to be representative in terms of observable characteristics.
An important feature of the panel is that participants agreed to be interviewed regularly, and the response rate is high throughout the study: it ranges between 76% and 92% in individual follow-up waves, and is 86% for the last wave, implemented at the end of November 2021. Of participants, 1,212 (58%) took part in all 12 waves of data collection: they form the ‘fixed sample’. Consequently, in the analysis, we report the main results for (1) all participants from the base sample who responded in a given wave, which we denote ‘full sample’, and for (2) the ‘fixed sample’, composed of individuals who participated in all 12 waves, eliminating the potential role of differences in samples across waves and making it easier to gauge the dynamics of treatment effects.
Information intervention
In wave 0, the participants were randomly assigned to either the Consensus condition (n = 1,050) or the Control condition (n = 1,051). In the Consensus condition, they were informed that the CMC conducted a large survey of almost 10,000 doctors from all parts of the country to collect their views on COVID-19 vaccines. They were also informed that the views were similar for doctors of different genders, ages and regions. Then, the participants were shown three charts displaying the distribution of responses of doctors regarding their trust in the vaccines, willingness to get vaccinated themselves and intentions to recommend the vaccine to their patients. Each of the charts was supplemented by a short written summary. The exact wording and the charts are provided in Section 3.3 of the Supplementary Information. In the Control condition, the participants did not receive any information about the survey of medical doctors.
Data
Before the information intervention in wave 0, we elicited prior beliefs about doctor’s views to quantify misperceptions about doctors’ opinions. Specifically, the participants were asked to estimate (1) the percentage of doctors in the Czech Republic who trust the approved vaccines, and (2) the percentage of doctors who are either vaccinated or intend to get vaccinated themselves. Later, in wave 1, we elicited posterior beliefs to estimate whether people in the Consensus condition actually updated their beliefs about doctors’ views based on the information provided. In each of the 12 waves, we asked respondents to report whether they got vaccinated against COVID-19. The main outcome variable ‘vaccinated’ is equal to one if the respondent reported having obtained at least one dose of a vaccine against COVID-19.
In the analysis, we report two main regression specifications: (1) a linear probability regression controlling for pre-registered covariates: gender, age (6 categories), household size, number of children, region (14 regions), town size (7 categories), education (4 categories), economic status (7 categories), household income (11 categories) and prior vaccination intentions, and (2) a double-selection LASSO linear regression selecting from a wider set of controls in Extended Data Table 1, including prior vaccine uptake and beliefs about the views of doctors.
Additional data to verify vaccination status
We collected two sets of additional data to verify the reported vaccination status in the main dataset.
Third-party verification
First, we used data collected for us by a third, independent party. We took advantage of the fact that different survey agencies have access to the panel our respondents are sampled from (the Czech National Panel). Although the main data collection was implemented by one agency (NMS), we partnered with another agency (STEM/MARK) to include a question on vaccination status in a survey implemented on its behalf among the same sample. As the survey agency, graphical interface and topic of the survey were different from our main data collection, we believe that respondents considered the two surveys to be completely independent of each other, and thus experimenter demand unlikely had a role in the second survey. The response rate was high (92.8%) and independent of the treatment (Extended Data Table 6). Out of 1,801 participants in wave 11, 1,672 also took part in the third-party verification survey implemented 2 weeks later. This allowed us to compare reported vaccination status at the individual level for a vast majority of our sample, and to test whether Consensus affects the level of consistency in reporting of being vaccinated across surveys.
Certificate verification
The second verification links the reported vaccination status with an official proof of vaccination. We exploited the fact that all vaccinated people receive an EU Digital COVID certificate issued by the Czech Ministry of Health, which was often used as a screening tool at the time of data collection. We collected the data on vaccination certificates among respondents from our full sample who (1) participated in wave 11, and (2) reported to have at least one dose of the COVID-19 vaccine in wave 11 (n = 1,414). We asked respondents whether they had the certificate with them. Of participants, 96% confirmed that they had the certificate with them, and this proportion is very similar across the Consensus and Control conditions (χ2(1,n = 1,414) = 0.999, P = 0.318). Those with a certificate were asked to type in several specific pieces of information about the applied vaccine that are unlikely to be known by someone without a certificate (for example, the correct answer for those who got a vaccine from Pfizer/Biontech is ‘SARS-CoV-2 mRNA’). Assessment of the typed text by independent raters suggests that, conditional on their having the certificate, more than 94% of respondents actually looked at the certificate when responding to our detailed questions. This rate is again very similar across conditions (χ2(1,n = 1,364) = 0.473, P = 0.492).
More details about both verification procedures and results are in the Supplementary Information.
Ethics approval
The research study was approved by the Commission for Ethics in Research of the Faculty of Social Sciences of Charles University. Participation was voluntary and all respondents provided their consent to participate in the survey.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this paper.
Data availability
The experiment and analyses were pre-registered on the AEA RCT Registry (AEARCTR-0007396). The dataset generated and analysed for the main experiment is available in the Harvard Dataverse repository (https://doi.org/10.7910/DVN/RH0T6R). The availability of the dataset from the supplementary survey with medical doctors is subject to the approval of the CMC.
Code availability
The code to replicate the analyses and figures is available in the Harvard Dataverse repository (https://doi.org/10.7910/DVN/RH0T6R).
References
Greenwood, B. The contribution of vaccination to global health: past, present and future. Phil. Trans. R. Soc. B 369, 20130433 (2014).
Banerjee, A. et al. Selecting the Most Effective Nudge: Evidence from a Large-Scale Experiment on Immunization. Working Paper 28726 (NBER, 2021).
Jarrett, C. et al. Strategies for addressing vaccine hesitancy—a systematic review. Vaccine 33, 4180–4190 (2015).
Milkman, K. L. et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc. Natl Acad. Sci. USA 118, 10–12 (2021).
Dai, H. et al. Behavioral nudges increase COVID-19 vaccinations. Nature 597, 404–409 (2021).
Blendon, R. J., Benson, J. M. & Hero, J. O. Public trust in physicians—U.S. medicine in international perspective. N. Engl. J. Med. 371, 1570–1572 (2014).
Shapiro, J. M. Special interests and the media: theory and an application to climate change. J. Public Econ. 144, 91–108 (2016).
Dixon, G. & Clarke, C. The effect of falsely balanced reporting of the autism-vaccine controversy on vaccine safety perceptions and behavioral intentions. Health Educ. Res. 28, 352–359 (2013).
Lazarus, J. V. et al. A global survey of potential acceptance of a COVID-19 vaccine. Nat. Med. 27, 225–228 (2021).
McDermott, A. Herd immunity is an important—and often misunderstood—public health phenomenon. Proc. Natl Acad. Sci. USA 118, e2107692118 (2021).
Serra-Garcia, M. & Szech, N. Choice architecture and incentives increase COVID-19 vaccine intentions and test demand. SSRN Electron. J. https://doi.org/10.2139/ssrn.3818182 (2021).
Klüver, H., Hartmann, F., Humphreys, M., Geissler, F. & Giesecke, J. Incentives can spur COVID-19 vaccination uptake. Proc. Natl Acad. Sci. USA 118, e2109543118 (2021).
Alsan, M. & Eichmeyer, S. Experimental Evidence on the Effectiveness of Non-Experts for Improving Vaccine Demand. Working Paper 28593 (NBER, 2021).
Ashworth, M., Thunström, L., Cherry, T. L., Newbold, S. C. & Finnoff, D. C. Emphasize personal health benefits to boost COVID-19 vaccination rates. Proc. Natl Acad. Sci. USA 118, e2108225118 (2021).
Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K. & Larson, H. J. Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat. Hum. Behav. 5, 337–348 (2021).
Boykoff, M. T. & Boykoff, J. M. Balance as bias: global warming and the US prestige press. Glob. Environ. Chang. 14, 125–136 (2004).
Van Der Linden, S. L., Clarke, C. E. & Maibach, E. W. Highlighting consensus among medical scientists increases public support for vaccines: evidence from a randomized experiment. BMC Public Health 15, 1207 (2015).
World Health Organization. COVID-19: an informative guide: advice for journalists (WHO, 2021).
Haidt, J. Why social media is terrible for multiethnic democracies. Vox https://www.vox.com/policy-and-politics/2016/11/15/13593670/donald-trump-social-media-culture-politics (2016).
Jennings, W. et al. Lack of trust, conspiracy beliefs, and social media use predict COVID-19 vaccine hesitancy. Vaccines 9, 593 (2021).
Bursztyn, L., Rao, A., Roth, C. & Yanagizawa-Drott, D. Misinformation During a Pandemic. Working Paper 27417 (NBER, 2021).
Bronchetti, E. T., Huffman, D. B. & Magenheim, E. Attention, intentions, and follow-through in preventive health behavior: field experimental evidence on flu vaccination. J. Econ. Behav. Organ. 116, 270–291 (2015).
Campos-Mercade, P. et al. Monetary incentives increase COVID-19 vaccinations. Science 374, 879–882 (2021).
Bursztyn, L. & Young, D. Misperceptions about others. Annu. Rev. Econom. (in the press).
Haaland, I., Roth, C. & Wohlfart, J. Designing information provision experiments. J. Econ. Lit. (in the press).
Bordalo, P., Gennaioli, N. & Shleifer, A. Memory, attention, and choice. Q. J. Econ. 135, 1399–1442 (2020).
Czech News Agency (CTK). Mezi lékaři je očkováno 88 procent, více než polovina lékařů má posilující dávku [88 percent of doctors is vaccinated, more than one half have booster dose]. České Noviny https://www.ceskenoviny.cz/zpravy/mezi-lekari-je-ockovano-88-procent-vice-nez-polovina-lekaru-ma-posilujici-davku/2129777 (2021).
Bradley, V. C. et al. Unrepresentative big surveys significantly overestimated US vaccine uptake. Nature 600, 695–700 (2021).
Bloniarz, A., Liu, H., Zhang, C. H., Sekhon, J. S. & Yu, B. Lasso adjustments of treatment effect estimates in randomized experiments. Proc. Natl Acad. Sci. USA 113, 7383–7390 (2016).
Moehring, A. et al. Surfacing norms to increase vaccine acceptance. SSRN Electron. J. https://doi.org/10.2139/ssrn.3782082 (2021).
Santos, H. C., Goren, A., Chabris, C. F. & Meyer, M. N. Effect of targeted behavioral science messages on COVID-19 vaccination registration among employees of a large health system: a randomized trial. JAMA Netw. Open 4, e2118702 (2021).
Cohn, A. & Maréchal, M. A. Priming in economics. Curr. Opin. Psychol. 12, 17–21 (2016).
Haaland, I. & Roth, C. Labor market concerns and support for immigration. J. Public Econ. 191, 104256 (2020).
Dunwoody, S. in Communicating Uncertainty: Media Coverage of New and Controversial Science (eds Friedman, S. M., Dunwoody, S. & Rogers, C. L.) 59–79 (Lawrence Erlbaum, 1999).
Schmid, P., Schwarzer, M. & Betsch, C. Weight-of-evidence strategies to mitigate the influence of messages of science denialism in public discussions. J. Cogn. 3, 36 (2020).
Milkman, K. L. et al. A 680,000-person megastudy of nudges to encourage vaccination in pharmacies. Proc. Natl Acad. Sci. USA 119, e2115126119 (2022).
Van Linden, S. L. D., Leiserowitz, A. A., Feinberg, G. D. & Maibach, E. W. The scientific consensus on climate change as a gateway belief: experimental evidence. PLoS ONE 10, e0118489 (2015).
Komenda, M. et al. Complex reporting of coronavirus disease (COVID-19) epidemic in the Czech Republic: use of interactive web-based application in practice. J. Med. Internet Res. 22, e19367 (2020).
Acknowledgements
We thank PAQ Research (especially D. Prokop) and NMS Market Research (especially L. Rambousek) for implementing the longitudinal data collection; STEM/MARK for implementing the third-party verification; the CMC, M. Kubek, D. Valášek, M. Matoušek, R. Mounajjed and J. Studený for implementing the supplementary survey of medical doctors and providing further statistics; and D. Cantoni, F. Matějka, G. Veramendi and J. Zápal for feedback. V.B. thanks the German Research Foundation (CRC TRR 190 and 444754857). M.B. and J.Chytilová thank the Czech Science Foundation (20-11091S). J.Cahlíková thanks the Max Planck Institute for Tax Law and Public Finance for generous funding of the project. This project has received funding from the European Research Council (grant agreement no. 101002898). The funders had no role in the design of the study, data collection and analysis, decision to publish or preparation of the manuscript. D. Korlyakova, T. Bielaková, M. Pospíšil, I. Burianová and D. Nováková provided excellent assistance.
Author information
Authors and Affiliations
Contributions
V.B., M.B., J.Cahlíková and J.Chytilová contributed equally to the preparation of the design, implementation and interpretation of the findings from the main experiment. V.B. initiated the project, organized and implemented the survey among doctors, and had a lead role in the data analyses. M.B. and J.Chytilová wrote the manuscript. J.Cahlíková secured most of the funding.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature thanks Dean Eckles and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
Extended Data Fig. 1 Comparison of development of vaccination rate in the Control group (Sample of adult Czech population) and the Czech adult population.
The horizontal axis represents a timeline. Population data means are for a Tuesday following the start of the data collection (Mondays) at a respective wave denoted by diamonds. The weighted Control group means are denoted by triangles. Control condition n = 800–1,051, depending on survey wave. Source of population data: Opendatalab, a website set up by the Faculty of Information Technologies at the Czech Technical University in Prague using open data from the Czech Ministry of Health (https://ockovani.opendatalab.cz/statistiky), ISSN 2787-9925 - http://aleph.techlib.cz/F/?func=direct&doc_number=000017426&local_base=STK02 (accessed on January 12, 2022)38.
Extended Data Fig. 2 Effects of the Consensus condition on the second dose uptake and on intentions to uptake a third (booster) dose (Main Experiment, Sample of adult Czech population).
This figure plots estimated treatment effects on 1) the second dose uptake (two doses were designed as a complete vaccination cycle for the most commonly used vaccines), and on 2) intentions to uptake a third (booster) dose. Markers show the estimated effects, the whiskers denote the 95%-confidence interval based on standard errors clustered at the individual level. Estimated effects and t-test (two-sided) p-values are reported in the Figure. No adjustments for multiple comparisons. Diamonds and triangles report estimates from a linear probability regression that controls for the pre-registered set of control variables. Squares and circles report estimates from a double-selection LASSO linear regression (dsregress command in Stata 17) selecting from a set of covariates in Extended Data Table 1. All regressions include wave fixed effects. In the upper part of the Figure we report the timing and control mean. We report estimates for the full sample (diamonds and squares) and for a restricted sample of respondents participating in all 11 waves (triangles and circles). Full sample: Consensus condition n = 807–904, Control condition n = 800–897, depending on survey wave. Fixed sample: Consensus condition n = 614; Control condition n = 598.
Extended Data Fig. 3 Weekly average of newly confirmed Covid-19 cases per 100,000 population.
Case data source: The Czech Ministry of Health (https://onemocneni-aktualne.mzcr.cz/api/v2/covid-19/osoby.csv, Accessed on January 12, 2022). Population data source: The Czech Statistical Office (https://www.czso.cz/csu/czso/obyvatelstvo-podle-petiletych-vekovych-skupin-a-pohlavi-v-krajich-a-okresech, Accessed on January 12, 2022) and ref. 38.
Supplementary information
Supplementary Information
This Supplementary Information file contains the following sections: 1. Supplementary Figure 1; 2. Supplementary Tables 1–9; 3. Supplementary Methods; 4. Populated Pre-Analysis Plan.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bartoš, V., Bauer, M., Cahlíková, J. et al. Communicating doctors’ consensus persistently increases COVID-19 vaccinations. Nature 606, 542–549 (2022). https://doi.org/10.1038/s41586-022-04805-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41586-022-04805-y
- Springer Nature Limited
This article is cited by
-
Field testing the transferability of behavioural science knowledge on promoting vaccinations
Nature Human Behaviour (2024)
-
Correlates of COVID-19 vaccination intentions and opinions about mandates among four groups of adults in South Africa with distinct vaccine intentions: evidence from a large national survey
BMC Public Health (2023)
-
COVID-19 vaccine uptake and vaccine hesitancy in rural-to-urban migrant workers at the first round of COVID-19 vaccination in China
BMC Public Health (2023)
-
Financial incentives for vaccination do not have negative unintended consequences
Nature (2023)
-
Determinants of COVID-19 vaccine fatigue
Nature Medicine (2023)