Introduction

According to Robert Dahl’s understanding of an “enlightened democracy,” people make political decisions in an informed way (see Mayer and Wassermair 2010, p. 173). However, public opinion research, in general, and research on voter decisions in direct democracy, in particular, have repeatedly argued that this is a very challenging goal, because in reality voters are neither capable nor willing to process all necessary information and, consequently, are most likely to make decisions based on low levels of knowledge or to rely on cues (Christin et al. 2002, p. 759; Colombo 2016; Colombo and Kriesi 2017; Neijens and De Vreese 2009).

We argue that vote advice applications (VAAs) can be an effective tool in our increasingly digitized context, because they provide voters with personalized information about their own positions vis-à-vis the positions of parties or policies. Like other decision aids (Neijens and De Vreese 2009; Neijens et al. 1992), VAAs do not only provide (new) information but also potentially help voters to more efficiently structure their opinions about an issue or a person. While research on the role VAAs play in voters’ opinion formation has increased over the years (Alvarez et al. 2014; Ladner 2012), it has almost exclusively focused on the electoral context, i.e., on situations when citizens vote for parties or candidates. In contrast, very few studies examine the role VAAs play in citizens’ opinion formation in direct-democratic decisions.

This study fills this gap by providing new insights into how VAAs affect individual decision-making in direct democracy both theoretically and empirically. More precisely, we ask whether and how the use of VAAs affects individuals’ opinion formation in a referendum context.

We draw on the literature on VAAs in the electoral context and on studies on individuals’ opinion formation in direct-democratic decisions to conceptualize a vote advice application as a “question and answer” tool. Users first answer questions about their stances on certain topics; the application then compares these stances to the positions of parties or candidates, or to specific issues (Alvarez et al. 2014; Ladner 2012; Sudulich et al. 2014). Ideally, the use of a VAA thus facilitates opinion formation and eventually even helps voters make decisions based on their “true” preferences.

The relevance of our study is at least twofold. First, it provides new insights into the role of VAAs in individuals’ opinion formation in the context of direct-democratic decisions. In contrast to elections, where most people have an initial idea about what party to vote for thanks to long-lasting party affiliations or some degree of closeness to a particular party, many voters enter direct-democratic campaigns in a state of relative ignorance (Converse 1964; Fishkin and Luskin 2005). This is reflected not least high shares of undecided voters at the beginning of a campaign (regarding the referendum under investigation in this study, see Dermont and Stadelmann-Steffen 2019). VAAs may therefore be even more important to voters’ opinion formation in such campaigns than they are in electoral contexts. All the more so because voting on ballot proposals may look like a simple yes/no decision but is the result of multidimensional choices. A specific ballot proposal consists of various elements; a voter may like some of those and reject others: she is faced with trade-offs. Decision making thus becomes a complex endeavor, whereby individuals require information to weigh the pros and cons of each proposal (Dermont and Stadelmann-Steffen 2019; Stadelmann-Steffen and Dermont 2018). The use of direct-democratic instruments is steadily growing around the world, which calls for additional research, especially as far as the mechanisms behind VAAs’ role in individuals’ voting decisions are concerned. Moreover, VAAs may also be attractive in contexts where direct-democratic instruments are used relatively rarely or have only recently been introduced, as they may compensate for the (still) lacking direct-democratic campaign structures in these environments (Heidbreder et al. 2019).

Second, we go beyond existing research by examining two potential mechanisms through which a VAA can influence voters’ opinion formation: The first relates to the mere use of a VAA, while the second refers to the message it conveys, i.e., the degree to which the user agrees with the policy at stake. Moreover, we differentiate between a persuasive effect (i.e., the idea that VAAs provide voters with new information or information that is different from what they would have at their disposal without the VAA), and an intensifying effect (i.e., the phenomenon of the VAA affirming voters’ original positions and therewith strengthening their vote intentions). It is worth noting that similar mechanisms are likely also at play during electoral campaigns (see Holbrook and McClurg 2005, p. 689, who refer to a similar mobilizing effect with respect to voter turnout), so our results may also be relevant to the role VAAs play in the electoral context.

We use novel data from the referendum campaign on the new energy law in Switzerland, which was collected in May 2017. Our analysis is based on a three-wave panel survey and a VAA experiment that was implemented in its last wave. In this third wave, which took place a week before the referendum, respondents were randomly assigned to a treatment and a control group. Only the former were shown the VAA and made to use it. A comparison of the treatment and the control groups provides evidence that the use of a VAA affects vote intention. Moreover, our analysis of respondents who had not yet voted by post (i.e., those respondents whose vote intentions could still be changed by their participation in the VAA) suggests that the VAA’s message could generate both a “persuasive effect” and an “intensifying effect,” depending on respondents’ original vote intentions and party affiliations.

Theoretical background

State of research

VAAs are designed to inform users about their optimal vote choice by supplying information about issues or party positions (Alvarez et al. 2014; Sudulich et al. 2014), thereby focusing on the outcome of decision-making (Price and Neijens 1998, p. 147f.). Existing VAAs and previous research on the topic have been almost exclusively limited to the context of elections, where VAAs “match” voters and parties. In so doing, VAAs usually compare voters’ and parties’ positions on numerous policy statements and indicate the degree of agreement between voters and parties. The selection of statements is pivotal, as the number of questions and their wording decide how well the different issues and their related aspects and positions are covered. Most VAAs also allow users to weight issues according to each issue’s perceived importance to the user. VAAs apply matching to calculate similarity scores (Mendez 2014) and existing research has demonstrated that, at least for salient issues, VAAs’ matching mechanisms work reasonably well (see Wagner and Ruusuvirta 2012).

The main questions surrounding VAAs thus come down to whether and how VAAs affect voters and their behavior. The optimistic account expects that VAAs guide users’ political opinions in independent and impartial ways (Holleman et al. 2016). The notion of proximity voting inherent to VAAs (Mendez 2014) is closely related to issue voting, which assumes that a voter’s proximity to the position of a party or an issue determines her vote choice (Garzia and Marschall 2019; Ladner 2012). The latter has become more relevant since the 1970s, when previously stable social cleavages started to lose importance. Furthermore, VAAs are theorized to strengthen the relationship between parties, citizens and (in some cases) the media, as a VAA positions itself squarely where these three players interact (Krouwel et al. 2014).

A more skeptical view, however, emphasizes that VAAs could foster unequal exclusion or “short-cut” decision-making. On the one hand, those unable to use the technology cannot benefit from VAAs; on the other hand, the voters who exclusively rely on VAAs for their decision-making reduce their democratic involvement to the last step of making a yes/no decision on an issue (Cedroni 2010). Furthermore, even though they bolster issue proximity, VAAs disregard other forms of representation, such as politicians’ and parties’ credibility and accountability (Wagner and Ruusuvirta 2012). More generally, existing research may have overestimated VAAs’ effects by failing to employ an experimental design and account for self-selection effects (Munzert and Ramirez-Ruiz 2021).

Empirically, a large number of studies focusing on different dependent variables, such as party preferences, party choice, and electoral participation, have been conducted in various countries and produced mixed results. While some have found VAAs to have no effects or only small effects (e.g., Enyedi 2016; Israel et al. 2016; Mahéo 2016; Marschall and Schmidt 2010; Ramonaite 2010; Walgrave et al. 2008), others have documented more meaningful and stronger relationships (Andreadis and Wall 2014; Christensen et al. 2021; Garzia et al. 2014; Garzia et al. 2017; Gemenis and Rosema 2014; Germann and Gemenis 2019; Kamoen et al. 2015; Kamoen et al. 2019; Ladner and Pianzola 2010; Munzert and Ramirez-Ruiz 2021; Pianzola et al. 2019; Ruusuvirta 2010; Van de Pol et al. 2019; Wall et al. 2014). In the 2007 Swiss elections, for example, VAA use had a self-reported influence on the vote choices of 66.5% (Fivaz und Nadig 2010) to 70% (Ladner 2012) of their users. In the Netherlands, more recent studies have shown that voters are more likely to choose a party when it has been recommended by a VAA (Kleinnijenhuis et al. 2019; Wall et al. 2014).

To the best of our knowledge, a single study has so far examined the influence of VAA use on a non-party vote, namely on voting in the Brexit referendum. According to this study, voters were more likely to vote to “leave the EU” if they received policy information indicating that they were indeed closer to the “leave” side of the referendum (Trechsel et al. 2017).

In the following, we delve deeper into the role VAAs play in individuals’ opinion formation in a direct-democratic decision by applying and further developing existing arguments and findings from VAA research focusing on the electoral context.

Theoretical argument and hypotheses

Based on existing research, we conclude that a VAA may affect an individual’s opinion formation through two different mechanisms. First, we expect that the mere use of a VAA can make a difference. Obviously, this is the basic idea of a VAA, which has received empirical support, e.g., in Switzerland (Ladner 2012), but it is also in accordance with the general expectation that receiving more information affects people’s opinion (Luskin et al. 2002). We should thus observe differences between the vote intentions of voters who use a VAA and those who do not use it. Second, the specific message a VAA conveys can also be significant. In this case, we would expect to see varying reactions to the VAA among its users, contingent on what the VAA’s recommendation looks like and how it relates to a voter’s original vote intention.

Moreover, we further theorize potential VAA effects and argue that the latter can be both persuasive (i.e., VAAs can affect voting intentions by providing voters with new or additional information about their own positions) and intensifying (namely, VAAs can affirm voters’ original positions and further encourage them to vote for a specific party—or, in the context of direct democracy, to cast a “yes” or a “no” vote on a proposal). We suggest that differentiating between these two effects may be particularly important in a referendum vis-à-vis an electoral context because ballot decisions are inherently more directly related to (new) issues. Every ballot vote addresses a specific issue and question, on which some voters may have some prior knowledge and positions, while others may enter the referendum campaign in a state of relative ignorance. It is logical to expect that VAAs affect these two groups of voters differently.

Both the use and the message of a VAA can theoretically generate a persuasive and an intensifying effect. If the mere use of a VAA has a persuasive or an intensifying effect, we should observe that voters who have used a VAA find it easier to form a clear opinion on the ballot proposal. As a result, the share of undecided individuals should be lower in this group than in the group of voters who have not used the application. Furthermore, regardless of whether it recommends a “yes” or a “no” vote, the VAA can be expected to influence the final vote intentions of those who use it, either through its intensifying effect or through its persuasive effect.

Based on these considerations, our first two general hypotheses are:

H1

VAA use decreases the likelihood that a voter remains undecided about a ballot proposal (usage effect).

H2

The stronger a VAA message is, the more likely an individual is to follow it (message effect).

In an attempt to delve deeper into the effect of a VAA’s message—i.e., to differentiate between the aforementioned persuasive and intensifying effects—we consider that the role of the VAA is moderated by an individual’s original vote intention and whether the VAA recommendation is in accordance or in conflict with it and with party cues.Footnote 1

We first focus on those voters who do not have a clear original vote intention. Studies have shown that the more surprising a VAA’s recommendation is, the higher the likelihood of the voter changing their vote choice is (Ladner et al. 2012; Vassil 2011). This is in accordance with findings from research using Choice Questionnaires, indicating that individuals without a clear prior opinion are most reactive to new information (Bütschi 2004, p. 317f.; van Knippenberg and Daamen 1996). Undecided voters are, by definition, either not yet sure how to vote or simply have not informed themselves. A “surprising” VAA result—one that provides some new information about their positions—might therefore be particularly influential (Neijens and De Vreese 2009; Zaller 1992). For these voters, the persuasive effect manifests in the VAA’s message replacing their prior state of being undecided. Indeed, existing research offers empirical evidence that initially undecided voters are especially likely to cast their votes in accordance with their VAA results (Ruusuvirta 2010). These findings also lend support to the view that as the relevance of cleavage voting shrinks, VAAs could become an important means of overcoming voters’ lack of party identification (Mahéo 2016) and predispositions. The corresponding hypothesis thus reads:

H3

The association between the VAA’s message and voters’ final vote intentions is stronger for initially undecided voters than it is for voters with an original vote intention (persuasive message effect).

Individuals with an original vote intention can have the VAA result either confirm or challenge their original vote intentions (Vassil 2011). We discuss both scenarios.

If the advice matches the voter’s original vote intention, the user should perceive this as a confirmation of their initial intention, which should, therefore, be intensified. These dynamics are in line with the theory of “motivated reasoning”: People have directional goals they want to achieve; therefore, they tend to seek information that confirms their preexisting opinions in an effort to avoid cognitive dissonance (see Festinger 1962). Not all studies have been able to identify such effects but many have, and they conclude that whenever VAA results match voters’ original vote intentions, voters are further inclined to vote accordingly (Neijens et al. 1992; Talukder et al. 2021; Wall et al. 2014). These dynamics played out in the study about the use of VAAs in the context of the Brexit referendum: Those who received information that confirmed their original vote intentions were more likely to act on these intentions (Trechsel et al. 2017).

Prior vote intentions are often linked to an individual’s party affiliation or party preference. In this context, recent research on direct-democratic decisions has examined how party heuristics and specific policy information interact (Colombo and Kriesi 2017; Dermont and Stadelmann-Steffen 2019). Theoretically, party and policy information can be conceptualized as two different modes of information processing. Whereas systematic processing entails that one thoroughly understands and evaluates all (policy) information available, heuristic processing requires much less cognitive effort or motivation. For example, one can simply rely on one’s party position to reach a decision. Additionally, two principles underpin this model: the “least effort principle” and the “sufficiency principle” (Chaiken and Ledgerwood 2012). The former suggests that individuals try to form their decisions as efficiently as possible, investing minimal effort, time, and cognitive resources in reaching conclusions. The “sufficiency principle,” however, states that individuals strive to enhance their judgmental confidence and therefore put a great deal of effort into thinking about an issue. Furthermore, heuristic and systematic information processing can interact to influence evaluations (Gawronski and Creighton 2013). In this context, VAA results can be seen as (easily available) policy information. If the individual consults a VAA, she gathers and evaluates more information about her personal attitudes. Doing so could refer to a more systematic dimension of information processing, as the VAA provides cognitive arguments. Similar to our reasoning about voters’ original vote intentions, their preferred parties’ positions on a direct-democratic ballot proposal matching the VAA’s output can also be expected to strengthen their decision to vote accordingly.

This discussion leads to the following hypothesis:

H4a

Congruence between the VAA’s message and respondents’ original vote intentions is associated with intensified original vote intentions (intensifying message effect).

H4b

Congruence between the VAA’s message and respondents’ preferred party’s position is associated with intensified original vote intentions (intensifying message effect).

The situation is more complicated if the VAA’s advice does not match voters’ original vote intention or the party cue. When this is the case, two reactions are possible: The users could either ignore the new advice or consider it and possibly change their vote choice. The former scenario is consistent with motivated reasoning and the idea that individuals tend to disregard conflicting information (see Kunda 1990) and avoid cognitive dissonance (see Festinger 1962). Similarly, individuals with strong prior attitudes are harder to influence (e.g., Neijens and De Vreese 2009; Zaller 1992). The latter relates to Dahl’s (1989) understanding that individuals are rational information processors who choose information based on accuracy goals and with the goal of reaching an enlightened decision. If individuals hold prior opinions about an issue, they use these opinions to “anchor” their evaluations of the new information available to them, which, if credible, is then used to update these prior opinions (Taber and Lodge 2006). When a conflict between the VAA’s information and voters’ prior vote intentions emerges, some individuals will eventually decide to change their vote choice, while others will retain their prior intentions. Hence, while both reactions suggest varying ways of handling VAA information, we assume that, on average, a VAA result that challenges a prior vote intention will reduce the likelihood of the voter eventually voting according to her original intentionFootnote 2:

H5

Incongruence between the VAA’s message and voters’ original vote intentions is associated with a higher likelihood of them changing said vote intentions (via-à-vis respondents who receive a congruent VAA recommendation) (persuasive message effect).

Research design

Case selection

Our case is a popular vote on a new energy law that took place in May 2017 in Switzerland. We argue that Switzerland in general and this vote in particular are suitable cases for our analysis for several reasons. First, Switzerland is home to the greatest number of direct-democratic decisions in the world (Altman 2010) and VAAs exist for all national and many subnational electionsFootnote 3 (but not for direct-democratic votes). This presents some analytical advantages. Even though VAAs are not available for direct-democratic votes, the idea of using VAAs is common and citizens are used to it. Moreover, carrying out the VAA experiment during a real direct-democratic campaign makes the VAA exercise in the survey even more realistic. Overall, this setting benefits both the internal and the external validity of the study. Second, a single national decision was taken in May 2017, which rarely happens in Switzerland. Campaigns on different issues interfere with one another if several proposals are on the ballot on the same day. In contrast, individuals’ decision-making on this specific issue can be analyzed in isolation, which allows us to rule out any interferences by parallel campaigns.

Overall, we argue that Switzerland in general and the ballot proposal on the new energy law in particular provide the ideal conditions for us to investigate our hypotheses as far as internal validity is concerned. However, we do acknowledge the need to be cautious when we generalize our findings to other contexts, votes, or issues. In particular, policy information and, thus, VAA information is, of course, specific to each ballot question. Hence, we cannot rule out that our results are case-specific—i.e., vote- or issue-specific—as well.

Data

This analysis is based on a three-wave panel survey. The first wave collected 2891 responses, the second wave—1841, and the third wave—1253. Overall, we have full data on 1,181 respondents who participated in all three waves. The three waves were conducted 10 weeks, a month, and a week before the vote, respectively. The first wave took place in a pre-campaign setting. At the time of the second wave, the campaign had just started to get into its “hot” phase, obvious in the start of paid media (newspaper) advertisements. It also generally coincided with the distribution of the postal ballots to all citizens. The ballot boxes for elections and popular votes close on Sunday at 12 p.m., but most voters return their ballots by mail before Sunday. The last wave took place in the last week of the campaign when many people had already cast their votes or reached a decision.

We used Qualtrics to collect the sample from online panels and targeted people over the age of 18 living in Switzerland. The sample used language, age, gender, and cantonal (subnational) unit quotas to be representative. The survey was carried out in all three official languages of Switzerland: German, French, and Italian. Nevertheless, recruiting enough Italian speakers for the repeated cross sections proved challenging, and as a result, the samples were only representative of the French and German parts of Switzerland. The sample was fairly similar to the Swiss population in terms of age and gender. Nevertheless, like in most surveys—the groups with low education and low income were somewhat underrepresented (FSO 2019, 2020).

The VAA treatment was implemented in the third wave. Respondents were randomly assigned into a treatment group and a control group; the VAA was used by the treatment but not by the control group. As Table 4 in the Appendix demonstrates, the treatment and control groups did not significantly differ in terms of respondents’ sociodemographic composition and original vote intentions (see also Table S2 in the Supplemental Online Information (SI)). This uniformity suggests that randomization worked well.

The treatment group was presented with ten statements about the new energy law (for further details, see Table 5 in the Appendix). Respondents could use a slider to indicate positions ranging from 0 (clearly disagree) to 100 (clearly agree) (see Figure S1 in the SI). Moreover, respondents had to weight the importance of each statement—they had to indicate whether a statement was “not important,” “of average importance,” or “very important” to them. The VAA score was then calculated as a weighted sum of these answers: the degree of agreement with each statement \(s\) was multiplied by its weight \(w\) (either 50 (“not important”), 100 (“of average importance”) or 200 (“very important”)).Footnote 4 This sum was then divided by the sum of the weights:

$$v= \frac{{\sum }_{i=1}^{10}{s}_{i}*{w}_{i}}{{\sum }_{i=1}^{10}{w}_{i}}$$

Respondents were then shown the individual percentage score of how much they agreed with the objectives of the law. The VAA score thus ranged from 0 to 100. Figure S3 in the SI depicts the distribution of the VAA scores. After receiving the VAA score, respondents were asked whether the result aligned with their expectations and whether it was relevant to them.

This setup implies that while the assignment to the VAA, its use in having respondents react to the ten statements, and their receiving their personal score make up an experimental setup, the treatment content, i.e., the VAA’s message, does not. In fact, the VAA recommendations shown to respondents naturally were realistic assessments of the latter’s positions based on the VAA items. Therefore, these assessments did not constitute a randomly defined information treatment.

Our main dependent variable is individuals’ vote intentions. We measured how respondents intended to vote in the referendum at the beginning of the survey in all three waves. In wave 3, we also asked whether they had already cast a vote (by postal voting). Moreover, an additional question at the end of the survey asked those respondents in the treatment group who had not yet voted by postal ballot about their vote intentions after receiving the information in the survey (namely, the VAA).Footnote 5 The question in wave 1 included the answers “yes,” “no,” “undecided,” and “don’t know.” In waves 2 and 3, everyone except those who had already cast their votes (i.e., the early voters in wave 3) received a question about their likelihood of voting “yes” or “no.” To compare vote intention over time, values below 31% in waves 2 and 3 were coded as an intention to vote “no,” values between 31 and 74% as “undecided,” and scores equal to or greater than 75% were considered an intention to vote “yes.” We chose these asymmetric thresholds to account for the effects of social desirability bias—namely, in order to avoid counting too many respondents as “yes” voters if they were not (quite) certain that they would actually vote “yes.” This coding resulted in realistic groups as per the pre-ballot surveys and, eventually, the ballot result.

Figure 1 shows voters’ mobility across the three waves, i.e., individuals’ vote intentions at different points in time, before any VAA treatment was set. The number of undecided voters decreased over time. At the same time, the proportions of those intending to vote in favor of and against the law, increased. Not unlike the actual referendum, in which 58.2% of the valid ballot papers were in favor of the law,Footnote 6 a majority of the respondents indicated that they cast a “yes” vote. A comparison of the first and the third waves suggests that the campaign helped individuals form an opinion on the issue—the number of undecided voters greatly decreased.

Fig. 1
figure 1

Vote intentions over the course of the campaign

Full size image

The figure depicts respondents’ vote intentions in each of the three waves and mobility across the three waves for the sample (n = 1181) before any treatment was set.

We faced the problem that when the VAA was deployed in the third wave of the survey, 280 respondents in the treatment group and 268 respondents in the control group had already cast a vote by post. These members of the treatment group were not asked about their vote intentions after the treatment, because they had already voted. Moreover, we can assume that these voters might have been less prone to react to the VAA anyways, since they would have wanted to hold on to the vote they had already cast. For this reason, our analysis excludes these early voters. Hence, the results we present in the following pages are based on the 633 respondents who indicated their vote intentions in all three waves and had not yet cast their votes. Out of these individuals, 326 (including 53 who did not have the right to vote) saw the VAA and were asked about their post-treatment vote intentions.Footnote 7

In the first empirical part, we rely on descriptive statistics and regression analysis to assess the effect of using a VAA (H1). We compared the treatment group’s vote intentions in wave 3 before and after using the VAA to the vote intentions of the control group in the same survey wave. Following H1, we hereby focus on whether the likelihood of being undecided differs between the treatment and the control groups.

In the second empirical part, we delve deeper into the mechanisms of the VAA’s message (H2-H5), concentrating on the treatment group, i.e., on those respondents who used the VAA. We employ OLS regression. We use the panel structure of the data and focus on within-individual variation in vote intentions—namely, on the change between a respondent’s vote intention in wave 1, which we consider her original vote intention, and her vote intention in wave 3 after she has used the VAA. We use respondents’ probabilities of casting a “yes” vote as a dependent variable and also account for their socio-demographics (education, age, sex, and household income), as well as their levels of political interest, their party identification, and party preferences (as measured in wave 1). To test the robustness of our results, we replicated the main models using respondents’ original vote intentions measured in waves 2 and 3, respectively, instead of those measured in the first wave. In fact, given that people’s vote intentions varied over the course of the campaign (especially among the initially undecided), the wave to which we choose to compare respondents’ final vote intentions may influence our results. At the same time, our theory does not inform us which comparison is most relevant to our study. Using the first wave, as we did in our main analyses, enables us to specifically investigate how initially undecided voters reacted to the VAA. Conversely, opting for wave 2 or 3 would probably allow us to distinguish the effect of the VAA from other campaign effects. We therefore present these models in the Supplemental Information (Tables S4 and S6).

More information about the variables, their operationalization, and their distributions can be found in Table 4 in the Appendix.

Empirical results

Testing the effect of using a VAA—comparing the vote intentions of the control and the treatment groups

Figure S4 and Table S2 in the SI document the self-reported voting intentions of the treatment and the control groups over the three waves. We observe that the two groups of voters were very similar in their vote intentions in waves 1 and 2. In the third wave, the treatment group was slightly more likely to oppose the ballot proposal than the control group (25% vs. 22%), before completing the VAA, however, this difference proved not to be statistically significant in an ordinal logistic regression. While the group of undecided voters was slightly larger in the control group, approximately 33% of the respondents in each group were in favor of the law. Presaging the actual ballot outcome, the “yes” voters formed the majority in both groups. Overall, these data suggest that the treatment and the control groups were similar before the treatment. We proceed to look at the effect of using a VAA, and more specifically at whether, as H1 posits, completing the VAA decreased one’s likelihood of being undecided.

Table 1 presents the results of the logistic regression models. The dependent variable is the likelihood that a respondent is undecided in wave 3. We use two slightly different measures. Models (1a) and (1b) use the vote intentions of the treatment and the control groups at the beginning of wave 3, i.e., before the VAA treatment. Those who indicated that they were undecided were coded 1 and all others are coded 0. We apply the same binary coding in Models (1c) and (1d), but this time we use the vote intentions of the treatment group after the treatment.

Table 1 The effects of VAA use on the likelihood of being undecided in wave 3
Full size table

The main explanatory variable in all models is the use of the VAA—whether a respondent was in the treatment group and, thus, completed the VAA. Moreover, Models (1b) and (1c) control for respondents’ vote intentions in wave 1 and include an interaction between VAA use and vote intention in wave 1 to allow for the possibility that the effect of using a VAA is contingent on respondents’ original vote intentions. These models would confirm a significant treatment effect if there were a significant VAA effect in models (1c) and (1d) but not in models (1a) and (1b). We would interpret such results to mean that the likelihood of being undecided does not differ between the treatment and the control groups before the treatment, but a significant difference between the two does emerge after the treatment.

Empirically, Models (1a) and (1c) do not lend support to such a pattern. The VAA variable is not statistically significant in either model. Models (1b) and (1d), however, reveal that when we account for whether individuals were undecided in wave 1, the effect of using the VAA (i.e., the treatment effect) differs significantly between respondents who were undecided in wave 1 and those who already had a vote intention at that early stage of the campaign. In fact, among the initially undecided, the likelihood of still being undecided shortly before the referendum was lower in the treatment group than in the control group. Conversely, individuals who had already indicated a vote intention in wave 1 exhibited a slightly higher likelihood of being undecided in wave 3 if they completed the VAA (however, here, the difference between the treatment and the control groups was not significant). The right panel of Fig. 2 illustrates the interaction effect using the post-treatment vote intention, while the left panel plots the interaction effect for the pre-treatment vote intention. The comparison of the two plots reveals that the overall patterns are similar in both cases. Thus, while the difference among the initially undecided is only significant after the treatment, which corroborates a VAA usage effect, the similar patterns imply that this treatment effect should not be overrated.

Fig. 2
figure 2

The effect of VAA use contingent on original vote intention. Notes: Predicted values based on Models (1b) (left panel) and (1d) (right panel)

Full size image

Testing the effect of the VAA’s message within the treatment group

We proceed by delving deeper into the VAA’s message mechanism. The OLS estimations (Table 2, Models 1–3) initially lend support to H2: The more the VAA result bolsters a “yes” vote (or a “no” vote, respectively), the likelier an individual is to vote “yes” (or “no,” respectively). Hence, the VAA revealing that respondents’ issue-related attitudes are highly consistent with the ballot proposal is associated with a stronger post-treatment intention to vote “yes.” This pattern holds even when we control for original vote intentions.

Table 2 Linear regression models: probability of intending to vote “yes”
Full size table

As far as respondents’ original vote intentions are concerned, unsurprisingly, an original intention to vote “yes” (measured in wave 1; the reference category is “undecided”) is positively associated with a propensity to eventually cast a “yes” vote. In contrast, the coefficient of an original intention to vote “no” is no longer significant when the model controls for the VAA scores (from Model 3 onwards in Table 2). This might be because the number of intended “no” voters was very small in the first wave.Footnote 8

Hypotheses 3, 4, and 5 go a step further by considering the interaction between respondents’ initial vote intentions and the specific message that the VAA conveys to those who use it. In this context, Figure S7 in the SI plots the degree to which respondents perceived the VAA result to be in line with their original vote intentions (Figures S5 and S6 in the SI compare the objective, rather than the subjective correlation between respondents’ VAA results and their original vote intentions, and confirm the following conclusions). According to Figure S7, a majority of respondents indicated that the VAA mostly confirmed their initial positions on the referendum proposal. It is unreasonable to assume that the VAA led these individuals to change their vote intentions (i.e., a persuasion effect); however, their certainty in their vote intentions could have increased (i.e., an intensifying effect, as suggested by H4).Footnote 9

Conversely, the VAA provided new information to a substantial minority (roughly 30% of respondents)—i.e., it showed them that their prior vote intentions possibly did not reflect their issue-specific preferences. More specifically, 17.2% had expected a weaker agreement with the energy law and 12% a stronger one (7.7% did not know what to expect). This means that we can expect the VAA to have potentially changed this group’s final vote intentions, i.e., to generate a persuasive effect, either by pushing formerly undecided individuals to adopt a “yes” or a “no” intention (H3), or by triggering a change in their vote intentions from an original “yes” to a later “no,” and vice versa (H5).

While these descriptive results suggest that all three hypotheses have potential, we proceed to test their specific mechanisms. Model (4) in Table 2 corroborates that the VAA’s message effect is indeed contingent on whether an individual had an original vote intention.

Figure 3 plots this interaction and lends (only) partial support to Hypothesis 3—that originally undecided voters are most strongly affected by the message of the VAA. The slope is steeper for undecided individuals than it is for “yes” voters, but the relationship between respondents’ VAA results and their final vote intentions is not significantly weaker for those who had originally intended to vote “no” (for a similar finding and the use of an information and choice questionnaire, see Neijens and De Vreese 2009)

Fig. 3
figure 3

The VAA’s effect contingent on original vote intention. Note: Predicted probabilities calculated based on Model (4), Table 2

Full size image

.Footnote 10

These findings also provide a first indication that the VAA’s message can have both an intensifying effect (namely, a high VAA result can solidify an original intention to vote “yes”) and a persuasive effect (i.e., it can provide some new (potentially contrasting) information, as H4 and H5 suggest). More explicitly, we test the validity of H4 and H5 in two steps. First, we follow H4 and look at the role of party affiliation (Models (6) and (7) in Table 2). Then, we delve deeper into the role of respondents’ original vote intentions (Table 2).

As far as party affiliation is concerned, Models (6) and (7) in Table 3 show that party affiliation and its interaction with respondents’ VAA results are indeed associated with final vote intentions. Overall, respondents who favor right-wing parties were likelier to reject the ballot proposal. Importantly, however, we observe positive interaction terms between respondents’ VAA scores and all party groups. Respondents who adhered to a party had a stronger reaction to the VAA’s message than those who did not have or indicate a party preference. A high VAA result thus greatly increased the likelihood of a respondent eventually voting “yes” on the proposal across the entire political spectrum (from left to the right).Footnote 11

All presented results are robust to the inclusion of additional sociodemographic and socioeconomic variables (Model (7)). Moreover, these variables increase the explained variance in the probability of a final intention to vote “yes” (with an adjusted R2 of 0.531).

To gain more insight into the interaction between respondents’ VAA results and their original vote intentions, we create a new variable that more specifically captures these two features. The VAA score was coded as: 0-30-“no,” 31-74-“undecided,” and 75-100-“yes.”Footnote 12 These VAA scores were then cross-tabled with respondents’ initial vote intentions, which resulted in a variable with nine possible outcomes: “no (vote intention) & no (VAA result),” “no & undecided,” “no & yes,” “undecided & no,” “undecided & undecided,” “undecided & yes,” “yes & no,” “yes & undecided,” and “yes & yes.” All outcomes could be observed in our sample, except the “yes & no” combination (an original intention to vote “yes” and a VAA recommendation to reject the proposal). The probability of a respondent intending to vote “yes” after receiving the treatment continues being our dependent variable.

Table 3 presents these regression results and confirms that the association between the VAA’s recommendation—its message—and respondents’ post-treatment vote intentions is significant. The more fine-grained analysis reveals that, depending on its result, the VAA can lead users’ opinions to move in both directions (toward both a “yes” and a “no” vote). As the results above and H3 would lead us to expect, we observe that the correlation between the VAA’s message and originally undecided voters’ final vote intentions is rather strong. Compared to undecided voters with an undecided VAA result, a VAA score below 31 is associated with a significantly and substantially lower probability to eventually vote “yes,” whereas a VAA outcome between 75 and 100 goes hand in hand with a significantly and substantially higher likelihood of accepting the ballot proposal. These results also hold when we control for age, sex, household income, and education. Hence, the findings confirm that the aforementioned aggregate trend toward a “yes” vote did not stem from VAA-induced biased opinion changes, but from the fact that many (undecided) individuals exhibited attitudes that came close to a “yes” vote and which were made visible to them by the VAA.

Table 3 Linear regression models —original vote intentions and VAA outcomes combined
Full size table

In a similar vein, the estimation results identify an intensifying effect for those respondents with congruent original vote intentions and VAA results. If a respondent intended to vote “no” and the VAA presented a similar result (i.e., < 31), the latter reinforced the former. Conversely, based on the model, when an original intention to vote “yes” is combined with a VAA score above 74, the probability of the voter declaring an intention to vote “yes” increases by more than 41 percent vis-à-vis the reference category. This lends support to H4.

Finally, our data do not allow us to provide systematic empirical support to the hypothesis that using a VAA leads to changes in respondents’ vote intentions, i.e., from an original intention to vote “no” to a final “yes” vote, or vice versa (H5). While no original “yes” voter received a really low VAA result, there was a small group of original “no” voters with moderate-to-high VAA results. However, group sizes are too small for us to identify significant differences.

Our robustness tests—when we use respondents’ original vote intentions from waves 2 and 3 instead of wave 1 (see SI, Tables S4 and S6)—largely confirm our results. Across all these models, the coefficient of the VAA’s message becomes somewhat smaller if we use respondents’ vote intentions at the later stages of the campaign, rather than in wave 1. This suggests that some changes in vote intentions stemmed from campaign effects, which might be related to the VAA outcome. For example, during the campaign, a previously undecided but right-leaning voter may be exposed to campaign information that pulls her in the direction of a “no” vote (i.e., provides the same directional information as a low VAA score). It is important to note that the VAA variable remains significant across all of these models. Hence, the outcome of the VAA remained a significant predictor of respondents’ final vote intentions even when we controlled for and considered voters’ vote intentions in the middle and at the end of the campaign.

Conclusions

While previous research on VAAs has almost exclusively focused on electoral decisions, this paper joins the very few studies examining the role of VAAs in direct-democratic votes. We have used data from a three-wave survey on the 2017 referendum on the new energy law in Switzerland to investigate how using a VAA and receiving its message affected respondents’ final vote intentions. The most important findings can be summarized as follows.

First, our analysis lends some empirical support for a VAA usage effect: However, while we do not observe a general treatment effect, we find that for the subgroup of initially undecided, the likelihood of still being undecided shortly before the referendum is lower in the treatment group than in the control group.

Second, our analyses quite consistently point to a VAA message effect. Most generally, when respondents see their VAA results, they tend to declare a final vote intention strongly aligned with these results. This pattern holds across different groups—namely original “no” voters, original “yes” voters, and formerly undecided respondents. The results provide consistent empirical support for the idea that a VAA can have both an intensifying effect (i.e., one that assures voters with original vote intentions that they have “gotten it right” and, thus, reinforces their intentions) and a persuasive effect mainly on formerly undecided voters (i.e., one that enables these respondents to form a vote intention by virtue of the VAA providing them with new or additional information).

Third, the main implication of these findings is that VAAs have the potential to facilitate voters’ opinion formation in direct-democratic votes. A large share of voters enters a direct-democratic campaign in a state of relative ignorance. This share reflects the large proportion of undecided voters at the beginning of the campaign. Our results demonstrate that these individuals can use the information from the VAA to reach a decision on the ballot proposal, which may, eventually, also increase the likelihood that these individuals participate in the vote. Similarly, like party cues, VAAs may help reduce the uncertainty even of voters with prior vote intentions by reassuring them that they have made the right choice on an often complex ballot proposal.

Our study is not without limitations. First, we have analyzed vote intentions, rather than actual (reported) voting decisions. Nevertheless, we assume that our setup constitutes a very conservative setting to test for VAA effects, because VAA information may be particularly relevant to last-minute decisions by as-of-yet undecided voters. Second, we have focused on a single referendum in a single country (and have thus analyzed one particular VAA on a specific topic). Therefore, we cannot rule out the possibility that the results are driven by our chosen case. In particular, while on aggregate, our VAA users shifted toward a “yes” vote, we have no way of knowing whether the latter resulted from our VAA instrument or whether policy information increased popular support for ballot proposals on environmental and energy policy (see Dermont & Stadelmann-Steffen 2019). Hence, we could confirm that our VAA facilitated users’ opinion formation, as suggested by the VAA literature (Alvarez et al. et al. 2014; Sudulich et al. 2014), but we would not go as far as saying that it made voters find their “true” or optimal vote choice. Further research should therefore analyze the use of VAAs across different ballot decisions and topics. A third limitation has to do with the placement of the VAA shortly before the actual vote. While we sought to present the VAA to respondents at a stage when they had already reflected on the issue on their own, the disadvantage of our timing was that there were many early voters for whom we could no longer observe a change in intentions as a result of the VAA. Even when these changes were observed, they were short term, and we could not be sure that the VAA’s effect would persist over a longer period of time (i.e., for days or weeks). Fourth, our study also revealed some more general challenges related to the study of VAAs. The VAA results of a majority of respondents agreed with their original vote intentions. While this fact might be interpreted as a validation of the VAA instrument, it also means that for many respondents, a VAA could not conceptually lead to a behavioral change (i.e., a persuasive message effect for voters with original vote intentions). Empirically, this singles out a small number of cases in which VAA studies might be most interesting—namely where the VAA result diverges from an individual’s original vote intention and/or preferred party position, but where—as in our study—a lack of data does not allow for further investigation.

Nevertheless, we argue that our study is relevant despite these limitations and beyond our specific case. Most important, it documents, both theoretically and empirically, the need and relevance of investigating the role of VAAs in a direct-democratic context. We show that, compared to an electoral setting, it is particularly important to differentiate between VAA use and VAA message but also between a persuasive and an intensifying effect. More substantively, our results specifically point to the important interaction between original vote intention (or the absence of it), party cues, and the VAA. Future research should theoretically distinguish group-specific VAA effects and also collect larger comparative samples that allow for more fine-grained empirical analyses that consider context-specific variation.