1 Introduction

Elections are among the most important societal activities in democracies. The use of the “correct” electoral rule is, therefore, of huge importance, because, in general, the choice of the rule matters [see, e.g., Riker (1986), Kaminski (1999), Evci and Kaminski (2021)]. In this paper, we provide an empirical analysis of the regional parliamentary elections 2019 in the Austrian region of Styria. In line with many empirical findings in the literature, we provide additional empirical evidence that the choice of the voting rule has an impact on the outcome of the election. In addition, we analyze the consistency of preference statements under different voting rules and consider the aspect of strategic voting in the context of the above-mentioned elections.

One of the first significant studies on the impact of electoral systems on election outcomes has been provided by Duverger (1951). In particular, Duverger hypothesizes that electoral systems based on plurality lead to two-party systems, whereas rules based on proportional representation tend to create many-party systems. His explanation of how the number of parties in a country depends on a country’s electoral system has later been called “Duverger’s Law” in the literature. A further empirical approach can be found in Rae (1971), who carefully studies different electoral systems and their impact on parties. Inspired by this approach, Lijphart (1994) systematically investigates the political consequences of electoral systems in 27 democracies trying to reduce the confusion surrounding electoral systems.

Given the theoretical observation and empirical evidence that electoral rules do indeed matter, electoral engineering gained a certain amount of importance. Riker [see Riker (1986), (1988)] provides examples of political manipulation by means of changing the used voting method in order to gain an advantage, including the Third Force in France. Taagepera and Shugart (1989) analyze examples of unwarranted changes in voting rules and their impact on the respective countries. In addition, they provide certain guidelines for justified changes in electoral systems. Empirical investigations of the impact of electoral engineering have been studied, e.g., in Poland (Kaminski, 1999, 2002), where the design of voting rules for the post-communist area has been very illuminating. In particular, it has been shown that the impact of voting rules on the electoral outcome has not been perfectly understood by the relevant agents, leading to unintended disadvantages (i.e., a lower number of seats) as a result of changing the voting system. More recently, Evci and Kaminski (2021) provide a similar analysis about unintended political consequences of political engineering for Turkey. McCune and McCune (2022), however, provide an empirical study of American single-winner elections and conclude that in these elections, apart from the plurality rule, the choice of voting method rarely matters. Note that in contrast to that study, we are concerned with a regional parliamentary election using proportional representation, and we are not exclusively concerned with the party ranked first in the social outcome under a specific voting rule. The contribution of institutional factors and institutional choices to the transition and consolidation of new democracies in the developing world has been studied by Elklit and Reynolds (2002). Finally, there also have been studies examining the influence of linguistic aspects on voter behavior (Gerber et al., 2016) and party-related primacy effects because a candidate is listed first on a ballot (Flis & Kaminski, 2022).

A slightly different empirical approach can be found in the literature on behavioral social choice, which provides a comparison of the theoretical behavior of rational agents with how real decision makers in fact behave. Contributions such as Regenwetter and Tsetlin (2004), Regenwetter et al. (2006), Regenwetter et al. (2007) or Popov et al. (2014), analyze the consequences of theoretical results for the practical use of social choice rules, raising doubt about the empirical importance of certain negative results such as those on Condorcet cycles.

In recent years many experimental and survey studies have been conducted in connection with real-world elections. Perhaps the most relevant studies related to this paper are those by Baujard et al. (2021), Baujard et al. (2018), Baujard et al. (2014) on French elections, by Roescu (2014) on a Romanian election, by Alos-Ferrer and Granić (2012) on German elections and Darmann et al. (2017, 2019) on the 2015 regional parliamentary elections in Styria. In addition, there exists a substantial empirical and experimental literature in the field of political science which is concerned with elections especially in relation to strategic voting [see, e.g., Blais et al. (2005), Spenkuch (2015), Stephenson et al. (2018), and Blais and Degan (2019)].

In this paper, following our analysis of the 2015 parliamentary elections in the Austrian region of Styria (Darmann et al., 2017), we add an analysis of the regional elections in the year 2019. In contrast to the 2015 elections with eight parties on the ballot, in the 2019 elections only six well-known parties were running (with all of them eventually gaining seats in the parliament). Our research is based on an exit-poll survey—specifically on election day, in front of some randomly chosen polling stations, voters were asked to respond to various questions concerned with different ways to state their preferences by filling out a questionnaire. Those preferences have then been used in various ways.

First, we examine the social rankings of the parties under different voting rules and find that, for some parties especially, their positions vary considerably depending on the voting rule. Second, we use the data to analyze the properties and consistency of different versions of voters’ preferences as elicited by our survey. Third, we examine how the voting rules allow voters to behave strategically, leading to a discrepancy between elicited preferences and reported votes.

The paper is structured as follows: Sect. 2 introduces the survey design. Section 3 provides the hypothetical results under various different voting rules. This is followed by an analysis of the voters’ consistency in stating their preferences in different forms in Sect. 4. Section 5 discusses strategic behavior in the election and, finally, Sect. 6 concludes the paper.

2 Survey design and data

The data for this survey study were collected during election day, Sunday 24 November 2019, for the regional parliamentary election in the Austrian region of Styria.Footnote 1 Compared to other elections and previous elections in the region, the selected election was particularly interesting. There were exactly six parties running, all of them well-known and viable, i.e., with good chances to get seats in the parliament. Actually, all parties eventually did get enough votes to enter the parliament. To get a glimpse of voters’ preferences and feelings as closely connected as possible to the actual act of voting, we undertook an exit poll. In front of nine randomly chosen voting stations in the city of Graz, voters, when they left the polling station, were approached by our staff and invited to provide written and anonymous information about their preferences over the six parties running in the election.Footnote 2 The 937 voters who participated in the exit poll were given a two-page questionnaire that asked them to report which party they voted for in the actual election, to rank the six parties in order of preference, to indicate which parties they “approve”, to evaluate the parties on various scales, and to answer some demographic questions. A translated version of the questionnaire can be found in the Appendix.

When eliciting preference information, we intentionally avoided reference to any voting rule for two reasons: First, explaining such rules on the spot would be too complicated to ensure a deeper understanding. Second, a possible strategic behavior based on the voting rules seems implausible given that no detailed explanation of the voting rules were given and no real election (and therefore no consequences) is related to the participants’ preferences. Hence, we assume to have received the voters’ sincere preferences.

Our survey included about 5.5% of the voters in the nine randomly chosen polling stations. Given that the voters participated voluntarily in the survey, the raw data obviously has a certain participation bias. This can be seen in comparing the actual voting results with the declared votes by the participants. We observe a clear “liberal bias”, i.e., voters of more leftist/liberal parties such as Greens, KPÖ and NEOS were much more willing to participate in the survey. On the contrary, declared support for rightist parties, in particular for the FPÖ, was rather small, indicating that they either refused to participate, or were rather “ashamed” to declare their rightist attitude (although answers were anonymous and only in writing). We correct that bias by the use of weights; the weights were determined by dividing the shares for each party in the official result in those nine polling stations by the share the same party received in the exit poll.Footnote 3 For example, the SPÖ had a share of \(18.5\%\) of the votes in the official result and received a support of \(13.76\%\) in the exit poll. Therefore participants supporting the SPÖ were under-represented in the exit poll leading to a \(\frac{18.50}{13.76}=1.34\). If we compare the calculated weights, we observe that, in addition to the SPÖ, the ÖVP and the FPÖ also have weights larger than one and are therefore under-represented. The other parties are over-represented in the survey leading to weights smaller than one. Table 1 provides an overview over official and declared voting results in the nine polling stations.Footnote 4

Table 1 Official results, declared votes, and weights taking account the participation bias

3 Comparing voting rules

Given that our survey made no reference to voting rules, there was no incentive for strategic behavior. Hence, we assume all the preference information to be sincere. As will be shown later, strategic behavior does, however, occur in real elections [see, e.g., Blais and Degan (2017) and the literature cited therein]. This can be observed from the fact that, besides their preferences, participants were also asked to communicate for whom they had actually voted (and will be discussed in more detail in Sect. 5). When comparing the impact of different voting rules on the outcome, however, we will use exclusively the weighted (and sincere) preference data from the survey. Moreover, the current regional election was a parliamentary election. Hence, the goal of the parties was not purely to receive votes but to ensure that seats in the parliament are achieved. However, because it is not clear how to distribute seats based on any election outcome using more information than just top-ranked alternatives, we will stick to “social” rankings of the parties based on the hypothetically received votes only.

Obviously, our study faces certain limitations. In particular, given that we use the voters’ sincere preferences in determining outcomes under different voting rules, this does not lead to simulations of outcomes of actual electoral systems, because any strategic behavior based on a specific voting rule can of course not be observed. Hence, the comparison takes place purely under the hypothetical assumption of elections without any strategic behavior.

Our survey study is closely related to Darmann et al. (2017), in which the 2015 regional parliamentary elections in Styria were investigated. The current survey is different in three respects. First, in the 2015 elections, eight parties were running, including all six parties from the 2019 elections and, additionally, two very small parties (that, together, received less than 2% of total votes). Second, the 2015 survey included a slightly different set of preference elicitation methods, leading to some differences in the set of voting rules analyzed. Third, the survey in 2015 includes voters from all over Styria, whereas the current survey only includes voters from the city of Graz (the largest of four voting districts). Interestingly, the political situation changed quite drastically between the elections of 2015 and 2019. In particular, the overall election results showed massive changes with ÖVP (\(+7.60\) percentage points), Greens (\(+5.40\)), KPÖ (\(+1.77\)) and NEOS (\(+2.73\)) gaining, and SPÖ (\(-6.27\)) and FPÖ (\(-9.27\)) losing. Comparing the official results restricted to the nine polling stations used for the 2019 survey, we also observe a huge difference as displayed in Table 2 (the 2015 percentages are based only on the votes for the six parties stated).

Table 2 Official results in the nine polling stations 2015 and 2019

Reasons for the huge differences can be seen in various political developments between 2015 and 2019. The national government changed to a coalition of ÖVP and Greens (putting the Greens in the government for the first time). Moreover, the KPÖ has been consistently very successful in the city of Graz and the NEOS experienced significant support on a national level. All of that can partly explain those changes. Hence, it seems that comparisons between the survey studies in 2015 and 2019 are rather difficult to establish.

Concerning the hypothetical elections results, the 2015 study had a very consistent winner (SPÖ) and second place party (ÖVP). Also, the current study, as will be shown below, provided a very consistent outcome with the Greens winning in eight out of nine hypothetical elections. Based on the definitions in Darmann et al. (2017), the current survey illustrates the impact of voting rules using more elaborate preference information on medium and polarizing parties. Intuitively, a medium party is acceptable to a large proportion of the voters but induces strong positive or negative views only in small groups, whereas a polarizing party induces strong views (both, negatively and positively) in a large group of voters. As will be shown in this section, medium parties (such as the NEOS) will experience a positive impact from voting rules using more elaborate preference information, whereas polarizing parties (such as the FPÖ, but also the ÖVP) rather experience a negative impact.Footnote 5

In the following we want to introduce each of the nine voting rules which we applied on the preferences stated by the participants of the exit poll.Footnote 6 We selected those rules because of their practical or theoretical importance. The order of the voting rules is to a certain extent based on the complexity of the rule itself and the preference information needed.

3.1 Plurality rule

The plurality rule is used in many countries and is related to the proportional representation system used in the Styrian election and elsewhere in that voters vote for a single party (or candidate). The overall ranking of the parties is then determined by the number of votes received. If we assume sincere voting for the voters’ top-ranked party based on their stated preference rankings, the weighted vote shares are as displayed in Table 3.

Table 3 Outcome under plurality rule

Although the votes have been weighted accordingly, the results based on sincere voting differ considerably from the actual votes obtained by the parties at the corresponding polling stations (compare to Table 1). In terms of the ranking, the actual winner (ÖVP) is overtaken by the Greens. This means that a significant fraction of the voters did not vote for their most-preferred party and, hence, voted strategically. The extent to which this happened and certain explanations are discussed in Sect. 5.

3.2 Anti-plurality rule

Whereas the plurality rule allows one to vote for one party, the anti-plurality rule takes into account the objection of voters against parties, by requiring that each voter votes against exactly one party. The social ranking of the parties is then determined by the number of received (negative) votes in ascending order, i.e., the winner is the party with the lowest number of votes against. Because of its simplicity we considered this to be a rule of interest. For the weighted results in Table 4 we take, for each party, the share of lowest ranks in the stated preference rankings.

Table 4 Outcome under anti-plurality rule

Obviously, the anti-plurality rule favors parties which are perceived similarly to another party that is usually disliked more by most of the voters that oppose the party. Our data shows that this is the case for the ÖVP (in relation to the FPÖ), therefore making it the winner. Clearly, this is a very crude way to announce negative preferences. Evaluative voting (discussed below) offers a more subtle way to express this sort of opposition and, therefore, also leads to very different voting outcomes.

3.3 Plurality runoff

A very common rule, in particular used for the election of presidents (e.g., in France and Austria), is plurality runoff. One of the problems of the plurality rule is that the plurality rule winner could face quite intense objection from a large majority of the voters. Actually, in the case of n voters and k parties (with \(n \gg k\)), up to a fraction \((k-1)\frac{n}{k}\) of the voters could prefer every other party to the winning party. To avoid the situation in which a winning party loses all pairwise contests against every other party, the plurality runoff introduces a second round in which, if no party receives an absolute majority of the votes in the first round, the two parties with the highest number of votes under the plurality rule run against each other. Given the preference data from the exit poll, it can be seen from Table 5 that a majority of respondents rank Greens above ÖVP, so the plurality rule and plurality runoff outcomes are identical.

3.4 Single transferable vote

The single transferable vote (STV) applies the plurality rule sequentially until one party receives an absolute majority of the votes. That is, in every round the party with the fewest plurality votes will be eliminated and the voters of the eliminated party are allowed to recast a vote for another party in the next round. STV is of considerable practical importance and used in various elections, especially in Australia and Ireland. Usually the sequential process is not done by repeated elections but by asking for rankings in the first place, so that transfers are made according to the announced preference rankings.

Applied to the data from the exit poll, the first round would lead to an elimination of the NEOS, its votes would be transferred, and the process repeated until there existed a party receiving an absolute majority, which in this case are the Greens. Given our data, the ranking outcome is again identical to the plurality outcome.

Interestingly, although in principle we are using information about the full ranking of the individuals, parties which are often middle-ranked do not necessarily benefit, because they will be eliminated in earlier rounds given the rather low level of strong support (i.e., number of top-ranked positions). In our data this is true particularly for the NEOS when comparing its social rank under STV with its performance under various other rules discussed below.

3.5 Pairwise majority rule

Pairwise majority (or Condorcet) rule is included here since it is central to social choice theory (and can produce Condorcet cycles), in particular because it explicitly incorporates the democratic idea of winning majorities. It is based on pairwise comparisons between the parties, i.e., for each pair of parties we determine which party is preferred to the other party by a majority of voters. Hence, it is in principle based on ordinal information such as that provided by Question 2 in our survey and does not take intensities of preference into account. The weighted pairwise tallies can be found in Table 5, where the numbers indicate the percentages of weighted voters who prefer the row party over the column party (and the bold numbers indicate a majority).

Table 5 Pairwise tallies between parties, showing the percentages of voters preferring column party over row party, e.g., \(51.26\%\) of voters prefer SPÖ over ÖVP

Although there exists no Condorcet cycle, i.e., cycling pairwise majorities, the margins between SPÖ, ÖVP and NEOS are very small (between 1% and 3% of the total weighted votes). Hence, this preference profile is very “close” to containing such a cycle. The ranking of the parties under pairwise majority rule is stated in Table 6.

Table 6 Outcome under pairwise majority rule

Although the winner (Greens) is the same as under the plurality rule, the NEOS (ranked last under the plurality rule) do strongly benefit from such pairwise comparisons. However, as displayed in Table 5, the winning margins in three of their contests (against SPÖ, ÖVP and KPÖ) are very small and below 3% of the total weighted votes.

3.6 Borda rule

If full ranking information is available, a very common (and perhaps natural) way of using it is the Borda rule. This is, together with Condorcet’s pairwise majority rule, probably the most relevant rule in the theoretical literature and therefore of considerable importance in our study. It assigns pre-determined points to the different ranking positions, i.e., in the case of k parties, \(k-1\) points for every top rank, \(k-2\) points for every second rank, down to 0 points for being bottom-ranked.Footnote 7 Because the points are pre-determined, the rule is based purely on ordinal information not taking into account how intense the difference is between a higher ranked and a lower ranked party. Given the rankings from our data, the weighted share of the total points for the parties is displayed in Table 7.

Table 7 Outcome under Borda rule

Using full ranking information as input is especially harmful to polarizing parties, i.e., those parties that receive strong positive support from a considerable fraction of the voters but, at the same time, receive strong disapproval from a large fraction of the voters. The FPÖ is such a polarizing party and, compared to its result under plurality rule, is significantly worse off under the Borda rule. Also, the ÖVP remains in second position only by a very small margin. In addition, the Greens would again win in a Borda election, though the winning margins would be much smaller. But this is of no surprise because using Borda scores implies an upper limit for the difference in the vote shares of the different parties. This is also one reason for the rather close contest between the four parties ranked two to five. And, still, the NEOS (compared to plurality rule) could be seen as benefiting most from such a rule.

The following rules cannot be used with purely ordinal preference information. They need either different or additional information to determine the social outcome.

3.7 Approval voting

Another voting rule included in this study because of its importance, both in theory and practice, is approval voting. It allows voters to approve as many parties as they like. Each of them will receive one point and the ranking of the parties is determined by summing up the points over all voters. In our data, voters approved, on average, 2.12 different parties, the median number of approvals (as well as the mode) being 2. Only about \(25\%\) of the participants approved just one party. This underlines the problem with plurality rule where a voter’s choice has to be exactly one party, although more than one party seems acceptable for a vast majority of the voters. The weighted results under approval voting are given in Table 8.

Table 8 Outcome under approval voting

Again, the Greens get the largest number of approvals, whereas the FPÖ receives by far the lowest number of approvals, and can therefore be seen as being disapproved by most of the voters. The other four parties have very similar numbers of approvals. One interesting fact is that both the NEOS and the KPÖ do receive substantial approval, although voters do not vote for them if only one vote is possible. Hence, those parties clearly benefit when more information (in the form of approvals) can be communicated by the voters.

3.8 3-Scale evaluative voting (3-scale EV)

The previous rule, approval voting, can be seen as one special, and very simple, case of evaluative voting, i.e., those rules which are based on an independent evaluation of the individual parties by the voters. We consider it relevant to include those rules in our study, because individual valuations do play an important role in many practical decision processes such as cultural contests, sports competitions, rankings of products, and many other preference aggregation situations. However, the length of the scale on which the voters can evaluate the parties is important. Approval voting uses the smallest possible length, namely two. Although it is very simple to use, approval voting does not give voters the ability to distinguish between parties that they definitely disapprove and those about which they are neutral or uninformed. The simplest extension of approval voting taking into account this problem is 3-scale evaluative voting, which adds such a third category. Hence, it allows each voter to place the parties into one of three pre-defined categories according to the voter’s acceptance of the party. In the exit poll, the participants had the options “+”, expressing the voter’s acceptance of a party, “−” expressing that the voter finds the party unacceptable, and “o” expressing a neutral opinion about the party. To aggregate this information, for each “+” received, a party gets one point; for a “o”, it receives zero points and for a “−” it receives \(-1\) points. The sum of all points received determines the ranking of the parties. For the Styrian parliamentary elections the ranking, according to the weighted average points received, is stated in Table 9.

Table 9 Outcome under 3-scale evaluative voting

Table 9 shows that the Greens are the overwhelming winner under this rule, indicating that they are considered acceptable by a large part of the electorate. In contrast, the rule significantly harms the FPÖ, which was found unacceptable by the vast majority of the voters. In between we rather observe a very close contest, in particular between ÖVP, SPÖ and KPÖ, where only slight changes in the evaluations could have led to a different ranking of the parties.

3.9 Evaluative voting (EV)

In our exit poll, we also asked for evaluations along a much longer scale, i.e., giving the option to evaluate the parties along a scale from \(-20\) to \(+20\). To some extent, voters were asked to provide cardinal preference information, making it possible to express both intensity and indifference. Given the evaluations from the exit poll, the weighted results are stated in Table 10.

Table 10 Outcome under evaluative voting

Again, it is rather parties that do not generate strong feelings, such as the NEOS and the KPÖ, that benefit from this additional preference information. Possibilities to indicate stronger intensities in the preferences do obviously harm the ÖVP and the FPÖ.

Surprisingly, there was a clear trend to not use the full scale of 41 possible evaluations, but the focus was primarily on zero and any multiple of five. More than \(82\%\) of all the evaluations provided by the voters were among those nine numbers. Hence, the question arises whether an extended scale which, in principle, offers more freedom to the voters, is actually needed, because voters restrict themselves to just a small subset of possible valuations. It also suggests that the voters are not willing (or able) to communicate intensities in a very precise manner. Whether this is true in general seems to still be an open question.

3.10 Summary of voting rule outcomes

We have now discussed each of the nine voting rules applied to the preference data from the exit poll. Table 11 summarizes the hypothetical voting outcomes (in terms of social rankings) under those common voting rules, assuming sincere voting behavior.

Table 11 Outcomes of different voting rules

As we have seen in the definitions of the voting rules, to a certain extent the voting rules can be classified with respect to the preference information used. Up to the first single horizontal line, the rules mainly use information about the top (or bottom) ranked parties, in certain cases in a sequential manner. The second group requires information about the full preference ranking and the third category uses preference information beyond ordinal rankings, something which could be more or less difficult to provide.

We observe that the rankings derived from the nine analyzed voting rules show significant differences, in particular concerning the middle-ranked parties (see also Fig. 1).

Fig. 1
figure 1

Comparison of outcomes under different voting rules

The rather consistent winner seems to be in line with the recent literature on behavioral social choice [refer to Regenwetter et al. (2006)]. However, the significant changes in the middle positions can be seen important with respect to the relevance of electoral design and confirm theoretical and empirical results that the choice of the voting rule does in fact matter.

4 Consistency of preference elicitation

In an interesting contribution, Baujard et al. (2018) analyze the consistency among voting rules focusing on different types of approval voting. In particular, they show that under different voting rules, the announced preferences might, because of strategic considerations, be different. Obviously, our setting does not allow for such a comparison given that potentially used voting rules have not been communicated to the participants. However, given that the preference data available to us could be seen as being based on sincere voting behavior, we are able to analyze the consistency among different preference elicitation methods. It seems clear that ranking all of the parties is considerably more demanding than just voting for one party. Hence, the question arises whether voters would actually be able to provide such extended preference information. During the exit poll, voters were asked to which extent providing a full ranking of the parties was difficult or easy. Surprisingly, more than 75% of the voters who responded to that question considered this to be (rather) easy. Although the same question was not explicitly asked about the different forms of evaluative voting, at least for approval voting and 3-scale evaluative voting, the difficulty of providing that information does not seem to exceed the difficulty of providing a full ranking. This can also be seen in the percentages of voters that provided the necessary preference information for the various voting rules to be applied, presented in Table 12.Footnote 8

Table 12 Percentages of full responses required for the particular voting rules

Although our sample of voters is not representative, there is a clear indication that the intuitive differences in the difficulty of providing the varying preference information are similar to those perceived by the actual voters.

However, it is of considerable interest whether this apparent simplicity of stating different types of preferences also leads to consistent preferences, i.e., whether the preference stated in a more complex form can be translated into a preference of some less complex type. A simple first check could be seen in comparing approval voting with the Borda rule in the sense that an approved party should never receive a lower Borda score (i.e., be lower ranked) than a disapproved party. If that was not the case, this would imply inconsistent behavior by the voters. In a similar manner we compare approval voting with 3-scale evaluative voting and evaluative voting, i.e., we verify whether approved parties were given evaluations at least as high as disapproved ones. Table 13 shows the percentage of voters (who communicated the respective two types of preferences) that provided consistent preferences w.r.t. their approval votes. For example, \(93.78\%\) of the voters approved their top-ranked party.

Table 13 Consistency between different preference elicitation methods

Hence, if measured in this mild sense, as explained above, the voters have been very consistent at least with respect to their (more complex) preferences compared with their approval preferences. We can extend this to a comparison of the other three types of preferences, sticking to this mild comparison, i.e., only requiring that if a party a is considered at least as good as party b under preference type 1, then party a also has to be considered at least as good as party b under preference type 2. Insisting on strict preferences would drastically lower the rate of consistency and, if compared to the full rankings, where indifference could not be communicated, probably not correctly reflect the actual consistency. Table 14 provides the percentages of voters that consistently stated their preferences of the respective two types.

Table 14 Consistency between different preference elicitation methods

To conclude, the participants of the exit poll have been quite consistent in stating their preferences. Obviously, consistency was more likely to be observed when the dichotomous preference of approval voting was compared to any of the other types of preferences, and to be particularly high when extending the scale by just one degree. Most difficulties (and therefore errors) occurred when rankings and large-scale evaluations had to be provided. Here, the consistency level dropped to below two thirds. However, if one would still lower the threshold for consistency (e.g., by allowing for at most one mistake), further increases in consistency would occur.

Baujard et al. (2021) investigate, among other interesting aspects, a different kind of consistency, namely the impact of the length of the scale on evaluative voting and the use of extreme grades. They show that the distribution of the grades remains stable for different scale-lengths when reduced linearly to two classes. In addition, the use of extreme (negative) grades under different scales is very stable for polarizing parties. In our study, because of lack of respective data, a reduction to two classes does not seem appropriate. However, it is possible to analyze the consistency w.r.t. “extreme” valuations. For example, Fig. 2 shows the use of lowest grades in 3-scale EV and EV when voters did not approve a party. Because EV allowed for 41 different grades, we divided those grades into three roughly equally sized classes and used the lowest class with grades in the interval \(\left[ -20,-7\right]\) for the comparison (see Fig. 2).

Fig. 2
figure 2

Use of lowest grades in EV (i.e., grades in \(\left[ -20,-7\right]\)) and 3-scale EV (grade “−” yielding \(-1\) points) when not approved

Obviously, the 45-degree line indicates a perfect match between the evaluations using the two preference elicitation methods. As can be seen, the use of lowest grades (or lowest intervals in the case of EV) was very consistent. The evaluations for all of the parties lie close to the 45-degree line. Figure 2, however, also shows another interesting aspect. Large percentages of low grades in case a party is not approved indicate strong objection of the respective party. That is, parties which are far from the origin can be seen as polarizing or unpopular in the definition of Darmann et al. (2017). In the Styrian election, as seen in the figure, this is the case in particular for the FPÖ, but also to some extent for the ÖVP. In principle, the FPÖ is indeed perceived to be on the right of the political spectrum. The ÖVP has, historically, rather been seen as a popular party, i.e., one that is seen positively by a large part of society with strong support from a specific group of voters; however, in recent years it took some right-wing positions on certain political issues giving it a slightly polarizing character. On the other hand, medium parties (i.e., parties which are acceptable to a large fraction of society inducing strong views in small groups only) and popular parties would rather be close to the origin. Again this can be confirmed by looking at the NEOS and Greens in the picture. Of course, one could consider the interval for EV too large, but the picture does not significantly change if the interval is restricted to, e.g., \(\left[ -20,-15\right]\).

In addition, we could also look at the positive support and compare the different scales. Figure 3 compares the percentages of positive support under 3-scale EV and evaluations in the interval \(\left[ 7,20\right]\) under EV.

Fig. 3
figure 3

Use of grades in [7, 20] in EV and highest grade in 3-scale EV (grade “+” yielding 1 point) when approved

Again, the behavior of using highest grades is rather consistent given that all points are close to the 45-degree line. Concerning the types of the parties, the picture is not so clear. Very popular parties should of course be far from the origin, because voters that approve such a party should also evaluate it highly. This is indeed the case for the Greens. Medium parties, on the other hand, should not get, in general, such a strong support even if they are approved. This can be seen to hold for the NEOS and the KPÖ. On the other hand, for a polarizing party, one would expect also very strong support from those who approve this party. However, this is not the case for the FPÖ, although it can be classified as a polarizing party: roughly one quarter of the voters who approve the FPÖ do not give them the highest grades under EV and 3-scale EV.

In contrast to the negative grades, here the picture does change when the interval for EV is restricted to, e.g., \(\left[ 15,20\right]\), as shown in Fig. 4. Hence, the voters seem reluctant to give rather high evaluations for a party, which might indicate a limited level of satisfaction with the political landscape.

Fig. 4
figure 4

Use of grades in [15, 20] in EV and highest grade in 3-scale EV (grade “+” yielding 1 point) when approved

All parties do experience a drop in the share of highest grades in EV. This also indicates that voters do actually use the extended scale when evaluating the parties.

Finally, we can look at the voters’ behavior w.r.t. their grades distributed whenever parties have been approved or ranked highly. Table 15 provides some interesting comparisons about the voters’ preferences.Footnote 9

Table 15 Consistency comparisons under different preference elicitation methods

The first four lines of Table 15 show the percentages of voters who approve a party (AV(1)) and evaluate them with grades as specified (e.g., EV(>0) means that the party received an evaluation larger than 0, and EV(20) indicates that the party received an evaluation of 20). Obviously, the strongest support under EV, i.e., a grade of \(+20\), is only given by roughly one third of the supporters for popular and polarizing parties such as the Greens, ÖVP and FPÖ. Medium parties, as was to be expected, such as the NEOS and the SPÖ, receive a much smaller share of strongest support. Given that the median approval rate was two parties (see Sect. 3.7), one could also check (line 5) how approved parties have been ranked by the voters, i.e., whether they were among the top two ranked parties (rank(1,2)). Again, medium and rather non-polarizing parties might have been approved even from voters that do not rank them highly. Polarizing and popular parties which are approved, are higher ranked more often. The values in line 6 of Table 15 provide some information about how intensely the voters value their top-ranked party, i.e., which percentage of those voters that have a party top-ranked also assign them the highest possible grade. There is no clear tendency visible, but it seems that between one third and one half of the voters translate a top rank into the highest grade. This is rather surprising insofar as, theoretically, under EV it could not harm to always assign the highest possible grade to the party a voter likes most. Surely, because there was no real election based on those stated preferences, we would rather expect sincere behavior, and that is what the data seem to confirm. Finally, line 7 does the opposite, i.e., it shows how many voters that had a party bottom-ranked actually gave it a grade of \(-20\). Interestingly, negative feelings have been expressed much more intensely, especially towards polarizing parties such as the FPÖ and the ÖVP, where roughly two thirds of the voters that had them bottom-ranked also assigned them a grade of \(-20\).

5 Strategic voting

In addition to the previously discussed (in)consistency in a voter’s preferences, which rather depends on the complexity involved in stating one’s preferences, another discrepancy might depend on a voter’s intentional misstatement of her preferences. That is, the actual vote differs from the voter’s preferences. In terms of plurality rule, this would mean that a voter does not cast her vote for her highest ranked party. This is a widely discussed topic in the literature, usually termed strategic voting, which would deserve a paper on its own and therefore will only be briefly explained in this study for the sake of completeness.

Based on various famous theoretical contributions in political science [e.g., Duverger (1951) and Cox (1997)], recent years have shown a large increase in empirical literature on strategic voting [e.g., Blais and Degan (2017), Stephenson et al. (2018), Spenkuch (2018) and the cited literature therein]. Most of these empirical studies focus on single-winner plurality elections. However, in recent years more attention (using both surveys and lab experiments) has been given to proportional representation and mixed systems [see Blais and Degan (2019) for an overview]. Our analysis falls into the first category of survey-based approaches. One major advantage of our study is that our participants responded immediately after voting and in the same location, making the link to the actual election as close as possible. As has been pointed out before [see, e.g., Wright (1992) and Alvarez and Nagler (2000)], one major caveat can be seen in the fact that self-declaration of preferences is not without doubt. Obviously also in our data, certain announced behavior by some voters cannot be reasonably explained.

Most survey-based studies estimate the frequency of strategic voting to be within the range of 3% to 17% of the ballots cast [see Spenkuch (2013)]. Given our data, we can easily determine how many supporters of a party, i.e., voters that have this party top-ranked according to their announced preferences, did actually vote for a different party,Footnote 10 i.e., behave strategically. Going into more detail, one, at least initially surprising, fact is that, under plurality rule, the winner based on the stated preferences (Greens) is different compared to the winner determined from the declared official votes (ÖVP). This is a consequence of voters not voting for their most preferred party (as stated in their ranking) and, hence, indicates strategic voting. Out of the 820 participants in the survey that responded to those questions, 113 did not vote for their top-ranked party, i.e., we observe a share of roughly 14% of the voters who voted strategically.Footnote 11 There could exist various different reasons for a voter to vote strategically [see Blais and Degan (2017) for a more detailed discussion], depending on all sorts of information available to the voter when she casts her vote, i.e., pre-election polls, media coverage, etc. For example, one reason for a strategic vote can be seen in trying to avoid wasting one’s vote. If a voter’s top-ranked party has no chance to enter the parliament, then a voter might vote strategically for another party. A second reason might lie in the fundamental belief that a certain party (and therefore its distinctive concerns) should be represented in the parliament. Hence, if such a party might seem close to the threshold for entering the parliament, a voter might give her vote to such a party. Third, if the actually formed coalition after the election is important for a voter, then she might vote strategically to push a party to have enough seats together with the seats of her most preferred party to form a coalition, or to support a party to increase its strength in an expected coalition.

The previous reasons should give some arguments for strategic voting being rational. However, whatever reason for strategic behavior might be assumed, it does not seem rational for a voter to vote strategically for a very low ranked party, which did occur occasionally (10% of the strategic votes were for a party ranked in fourth position or lower). Hence, we would rather see this as irrational behavior (or, simply, a mistake) and, therefore, “reasonable” strategic voting in our data is observed for about 12.5% of the voters.Footnote 12 Table 16 shows how the supporters of a party (i.e., those that had the party top-ranked) did actually vote.

Table 16 Percentages of votes given to column party when row party is top-ranked

We observe that supporters of the Greens and the NEOS in particular did use their votes to a much higher percentage to act strategically. The reasons could, however, have been very different. According to all pre-election polls, the Greens were certain to get enough votes to enter the parliament. In addition, it was also very unlikely that they would be part of any coalition forming the regional government. Hence, supporters of the Greens did use their votes extensively to support either parties that were rather on the edge of entering the parliament (KPÖ) or those that could potentially be part of the government (SPÖ) and were considered ideologically closer to the Greens than other parties (such as the ÖVP or the FPÖ).

On the contrary, part of the supporters of the NEOS might have had the impression that their vote could be “wasted” given the chance that they might not get enough votes to enter the parliament (as has happened in the previous elections in 2015), allocating their vote to other parties, especially those for which seats in the parliament were certain (such as the ÖVP and the Greens).

Besides the behavior of voters of the respective parties, we can also check from which supporters a party actually received its declared official votes. This is displayed in Table 17.

Table 17 Percentages of received votes of the row party from voters having the column party top-ranked

Table 17 nicely shows the dynamics of the strategic voting. The KPÖ received a lot of votes from supporters of the Greens, which does intuitively make sense given that in pre-election polls the KPÖ was rather on the edge of entering the parliament and the general political orientation of Greens and KPÖ seems sufficiently similar. The same is the case for the NEOS with the additional fact that, given their more medium political orientation, they also attracted substantial votes from ÖVP supporters.

However, although most of the observed strategic behavior seems reasonable, given that the analysis is based on a single event, it does not give the possibility to state more general results.

6 Conclusion

In this paper we analyzed the 2019 Styrian Parliamentary elections based on data from an exit poll immediately after the actual election. We showed that, based on the obtained data, although the winner is stable under the nine voting rules considered, the social ranking is sensitive to different voting rules. This confirms many of the empirical studies in the behavioral social choice literature that find very consistent winners, but also the theoretical influence of voting rules on voting outcomes. Following Baujard et al. (2021) we were able to compare voting behavior under different preference query methods. It was shown that the use of evaluative voting under different grading scales was rather consistent. However, a comparison of preferences in terms of rankings and evaluations based on the \(\left[ -20,20\right]\)-scale showed considerable inconsistencies. In addition, we investigated the use of evaluative voting using longer scales, being able to show that this extended freedom in the evaluation of parties is only used in a limited way. Finally, a brief discussion of strategic behavior in the actual election has been provided. Although the general size of strategic behavior is, to a certain extent, in line with previous studies and the voters’ behavior could be reasonably explained, more general conclusions do not seem easy to draw because of the fact that data exists only for a singular event and has been obtained via self-declaration of the voters. In addition to deriving more general conclusions, an interesting direction for future research, however, could consist in parliamentary seat simulations [see Flis et al. (2020) and Evci (2020)] for each electoral rule.