Abstract
Ballot initiatives are consequential and common, with total spending on initiative campaigns in the US rivaling that of Presidential campaigns. Past work using observational data has alternately found that initiative campaign spending cannot affect initiative outcomes, can increase the number of votes rejecting (but not approving) initiatives, or can affect outcomes in either direction. We report the first field experiment to evaluate an initiative advocacy campaign with precision. We find that campaigns can influence both rejection and approval of initiatives by changing how citizens vote, as opposed to by influencing turnout or ballot completion. Our experiment (involving around 18 % of Oregon households in 2008) studied a statewide mail program conducted by a Political Action Committee. Results further suggest that two initiatives would have passed if not for the advocacy campaign to reject them. We discuss implications for theories about direct democracy, campaign finance, and campaign effects.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Research on the question of whether initiative campaigns matter could be viewed as a subset of work on the question of whether political campaigns matter, in general. Of course, this question has a long history, with some suggesting that campaigns can matter but only under some circumstances, and others arguing that campaigns have no effect (Jacobson 1978; Erikson and Palfrey 2000; Green and Krasno 1988). Recent evidence from randomized experiments or natural experiments suggests that campaigns can influence voter preferences (Gerber 2004; Arceneaux 2005; Huber and Arceneaux 2007), but possibly for only a fleeting period (Gerber et al. 2011).
Logistic regression yields similar results indicating balance.
The Democratic Performance Index (DPI) is a synthetic variable created by the National Center for an Effective Congress, designed to be broadly predictive of Democratic voting across political races. The index is designed to help political organizations make operational decisions about voter outreach campaigns. We use it as a covariate to improve the efficiency of our estimator, but give no causal interpretation to its coefficient. Since we use the DPI created before the 2008 election for use by political organizations during the 2008 campaign, we need not worry that this covariate is partially determined by the treatment and, hence, correlated with the treatment indicator.
By “robust” standard errors, we are referring to the so-called Huber Sandwich Estimator (c.f. Freedman 2006).
The Holm–Bonferroni method sorts the p values from a family of hypothesis tests from smallest to largest. The procedure finds the first p value in the list that satisfies \(p_{(k)} > \frac{ \propto }{m + 1 - k}\), where \(p_{(k)}\) is the kth ordered p value, and m is the number of tests in the family. Then it rejects all hypotheses up to k − 1. The method is less powerful than the Hochberg procedure, but does not require the hypotheses to be independent. In this situation the hypotheses are clearly not independent.
We cannot demonstrate, statistically, that the treatment effect in our experimental universe would generalize to the large precincts outside of our experiment universe. That said, we find no relationship in our experimental universe between precinct size and treatment effect. This suggests that the advocacy organization’s treatment may have resulted in similar average treatment effects in Oregon’s large precincts as we found in our experiment universe of small- and moderately-sized precincts.
Other research has suggested that ballot initiatives have a greater effect on turnout in lower-turnout elections (Childers and Binder 2012). This suggests that the absence of a finding for turnout may be a result of the relatively high turnout election, and that should be taken into account when considering the external validity of these results.
Factoring in the uncertainty of our estimates, we calculate a 95 % confidence interval of (1.07, 6.44).
References
Ansolabehere, S., & Hersh, E. (2010). The quality of voter registration records: A state-by-state analysis. Cambridge, MA: Department of Government, Harvard University.
Arceneaux, K. (2005). Using cluster randomized field experiments to study voting behavior. In D. Green and A. Gerber (Eds.), The science of voter mobilization: The Annals of the American Academy of Political and Social Science (vol 601, pp. 169–179).
Arceneaux, K. (2007). I’m asking for your support: The effects of personally delivered campaign messages on voting decisions and opinion formation. Quarterly Journal of Political Science, 2(1), 43–65.
Bowler, S., & Donovan, T. (1998). Demanding choices: Opinion and voting in direct democracy. Ann Arbor: University of Michigan Press.
Broder, D. S. (2000). Democracy derailed: Initiative campaigns and the power of money. New York: Harcourt.
Childers, M., & Binder, M. (2012). Engaged by the initiative? How the use of citizen initiatives increases voter turnout. Political Research Quarterly, 65(1), 93–103.
Converse, P. E. (2000). Assessing the capacity of mass electorates. Annual Review of Political Science, 3(1), 331–353.
De Figueiredo, J. M., Ji, C. H., & Kousser, T. (2011). Financing direct democracy: Revisiting the research on campaign spending and citizen initiatives. Journal of Law, Economics, and Organization,. doi:10.1093/jleo/ewr007.
Ellis, R. (2002). Democratic delusions: The initiative process in America. Lawrence: University of Kansas.
Freedman, D. A. (2006). On the so-called ‘Huber sandwich estimator’ and ‘robust standard errors’. The American Statistician, 60(4), 229–302.
Gabaix, X., Laibson, D., Moloche, G., & Weinberg, S. (2006). Costly information acquisition: Experimental analysis of a boundedly rational model. American Economic Review, 96(4), 1043–1068.
Garrett, E., & Gerber, E. R. (2001). Money in the initiative and referendum process: Evidence of its effects and prospects for reform. In D. Waters (Ed.), The battle over citizen lawmaking (pp. 73–95). Durham, N.C: Carolina Academic Press.
Gelman, A., & King, G. (1993). Why are American presidential election campaign polls so variable when votes are so predictable? British Journal of Political Science, 23(4), 409–451.
Gerber, E. (1999). The populist paradox: Interest group influence and the promise of direct legislation. Princeton, NJ: Princeton University Press.
Gerber, A. (2004). Does campaign spending work? Field experiments provide evidence and suggest new theory. American Behavioral Scientist, 47(5), 541–574.
Gerber, A., & Green, D. (2000). The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review, 94(3), 653–663.
Gerber, A. S., Kessler, D. P., & Meredith, M. (2011). The persuasive effects of direct mail: A regression discontinuity based approach. The Journal of Politics, 73(1), 140–155.
Green, D. P., & Gerber, A. S. (2008). Get out the vote: How to increase voter turnout. Washington, DC: Brookings Institution Press.
Holbrook, T. M. (1996). Do campaigns matter?. Thousand Oaks, CA: Sage.
Initiative and Referendum Institute. (2000). http://www.iandrinstitute.org.
Iyengar, S., & Kinder, D. R. (1987). News that matters: Television and American opinion. Chicago: University of Chicago.
Keane, L., Nickerson, D. (2013). A field experiment on nonpartisan mobilization and persuasion down-ballot. Unpublished manuscript.
Krosnick, J. A., & Kinder, D. R. (1990). Altering the foundations of support for the president through priming. The American Political Science Review, 497–512.
Lowenstein, D. H. (1982). Campaign spending and ballot propositions: Recent experience, public choice theory and the first amendment. UCLA Law Review, 29(3), 505–641.
Magleby, D. (1984). Direct legislation: Voting on ballot propositions in the United States. Baltimore, MD: Johns Hopkins University Press.
Meredith, M., Malhotra, N. (2008). Can October surprise: A natural experiment assessing late campaign effects.” Stanford Graduate School of Business Research Paper Number 2002.
Nicholson, S. P. (2003). The political environment and ballot proposition awareness. American Journal of Political Science, 47, 403–410.
Owens, J. R., & Wade, L. L. (1986). Campaign spending on California ballot propositions, trends and effects, 1924–1984. Western Political Quarterly, 39(4), 675–689.
Rogers, T., Gerber, A. S., & Fox, C. R. (2012). Rethinking why people vote: Voting as dynamic social expression. In E. Shafir (Ed.), Behavioral foundations of policy. New York: Russell Sage.
Rogers, T., & Nickerson, T. W. (2013). Can inaccurate beliefs about incumbents be changed? And can reframing change votes?
Samuelson, W., & Zeckhauser, R. (1988). Status quo bias in decision making. Journal of Risk and Uncertainty, 1(1), 7–59.
Schrag, P. (1998). Paradise lost: California’s experience, America’s future. New York: New Press.
Smith, D. A. (1998). Tax crusaders and the politics of direct democracy. New York: Routledge.
Smith, D. A. (2001a). Homeward bound?: Micro-level legislative responsiveness to ballot initiatives. State Politics and Policy Quarterly, 1(Spring), 50–61.
Smith, M. (2001b). The contingent effects of ballot initiatives and candidate races on turnout. American Journal of Political Science, 45(3), 700–706.
Smith, D. A., & Tolbert, C. J. (2004). Educated by initiative: The effects of direct democracy on citizens and political organizations in the American states. University of Michigan Press.
Smith, D. A., & Tolbert, C. J. (2010). Direct democracy, public opinion, and candidate choice. Public Opinion Quarterly, 74(1), 85–108.
Stratmann, T. (2005). Some talk: Money in politics. A (partial) review of the literature. Public Choice, 124(1–2), 135–156.
Stratmann, T. (2006). Is spending more potent for or against a proposition? Evidence from ballot measures. American Journal of Political Science, 50(3), 788–801.
Tolbert, C. J., & Smith, D. A. (2005). The educative effects of ballot initiatives on voter turnout. American Politics Research, 33(2), 283–309.
Tolbert, C. J., Grummel, J. A., & Smith, D. A. (2001). The effect of ballot initiatives on voter turnout in the American states. American Politics Research, 29, 625–648.
Weber, C., Dunaway, J., & Johnson, T. (2011). It’s all in the name: Source cue ambiguity and the persuasive appeal of campaign ads. Political Behavior. doi:10.1007/s11109-011-9172-y.
Acknowledgments
We thank Our Oregon for collaborating with us. We thank Josh Berezin and Kevin Looper for cooperation and assistance. We thank Analyst Institute and Catalist LLC for providing data. We thank Don Green, Max Bazerman, David Nickerson, Kevin Collins, Jennifer Green, and the Analyst Group for providing feedback. We thank Julia Kamin, Carly Robinson, and John Ternovski for help with analyses and editing.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix
Appendix
Table 7 summarizes available covariates for the treatment and control precincts. The reader will see that the treatment and control precincts exhibit good balance. Figure 1 plots on the x-axis the number of registered voters (as reported after the election) against on the y-axis their Democratic Performance Index (a synthetic variable produced in advance of the 2008 election that summarizes Democratic voting in previous elections from National Campaign for an Effective Congress). Squares represent Oregon precincts excluded from the study, triangles represent those included. The dividing line between the larger and smaller precincts can be seen clearly with included precincts (triangles) to the left and excluded precincts (squares) to the right. There are exceptions. The figure shows seven large precincts included in the study, represented by stray green triangles on the right. It also shows a number of smaller precincts excluded from the study, represented by blue squares on the left. We explain these two anomalous precinct types in turn.
Democratic performance by 2008 registration
The unexpectedly large precincts in the experimental universe, triangles on the right, resulted from discrepancies in the two sources we used to assess the number of voters per precinct—National Campaign for an Effective Congress and Catalist, LLC—and the administrative records that provided the outcome measures that we gathered after the election and matched. The values on the x-axis in Fig. 1 come from these post-election administrative records. Less than 1 % of the precincts in the experimental universe (seven) show this discrepancy. What is most striking about Fig. 1 is the sharp discontinuity between included and excluded precincts, providing confidence in our randomization, and our post-election merger of randomization and outcome files. Nonetheless, we reran the analysis described below omitting the seven strays as a robustness check and results were not altered.
Figure 1 also depicts a number of small precincts excluded from the experiment, squares to the left of the dividing line. These precincts were excluded for the reason previously described: the number of registrants reportedly in the precinct differed across the two pre-election sources by more than 20 %.
Our choice of experiment universe has little bearing on the internal validity of our study. Moreover, no experiment can represent all precincts, in all elections, and for all time. In that respect, this field experiments is not unique in being limited in its geographic, temporal and demographic extent. Nonetheless, we must account for the sample restrictions when considering external validity, particularly when speculating on the effects of the ballot guide campaign on voters in all of Oregon.
Table 7 presents summary statistics for both included and excluded precincts. The size difference between excluded and included precincts is evident. And while turnout rates are not dramatically different, excluded precincts are more Democratic as measured by the Democratic Performance Index. Figure 1 illustrates why this is true, there is a positive association between precinct size and Democratic voting. The association is no surprise; urban environments tend to have larger precincts and also greater numbers of Democratic-leaning voters. As an illustrative anecdote, consider that the most strongly Democratic county in Oregon, Multnomah County (which contains Oregon’s largest city, Portland), also had the highest rate of precinct exclusion in this study due to sizes—90 % excluded compared to 56 % statewide.
Post-election, four (0.6 %) of the 700 precincts included in the study (one control precinct and three treatment precincts) could not be matched to with the reported election returns. Again, we suspect that inevitable imperfections in available records are responsible for this, but by any standard this is a small loss-to-follow-up rate. To address the slight imbalance in the partisanship of treatment and control precincts, a stratified analysis was carried out, creating three strata using the Democratic Performance Index; for shorthand. We will refer to these as Democratic, Independent and Republican precincts. The balance in the strata is much better for the Republican and Independent strata, as evidenced in Table 8. The Democratic stratum still shows evidence of imbalance. This imbalance does not concern us, however, because dropping the Democratic stratum from the analysis, estimates become larger, not smaller, suggesting that this residual imbalance in the Democratic stratum cannot explain away the treatment effects.
Table 9 presents coefficients from regression models run with fixed effects for strata, along with interaction terms between strata and treatment assignment. For each row of the table, a p value from a test of the difference between the three effect estimates. (No “main effect” was included in the model so that three interaction terms, one for each of the three strata, could be specified and stratum specific treatment effects computed directly.) There are no significant differences between coefficients for any of the ballot measures regardless of the additional covariates included; in other words, we fail to find evidence of treatment effect heterogeneity. (Again the Holm–Bonferroni adjustment was used to control for family-wise error rate.) Combining these strata-specific estimates into “main effects” by averaging the three returns estimates that look very similar to the original estimates in the main body of the paper. Tables 10, 11 and 12 provide analogous results with fixed effects for strata for roll-off, candidate votes, and turnout, respectively. There are no significant evidence of treatment effect heterogeneity and “main effects” are generally in line with original estimates in the main body of the paper.
Rights and permissions
About this article
Cite this article
Rogers, T., Middleton, J. Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment Showing That They Are. Polit Behav 37, 567–593 (2015). https://doi.org/10.1007/s11109-014-9282-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11109-014-9282-4