Abstract
Risk-limiting audits (RLAs) for many social choice functions can be reduced to testing sets of null hypotheses of the form “the average of this list is not greater than 1/2” for a collection of finite lists of nonnegative numbers. Such social choice functions include majority, super-majority, plurality, multi-winner plurality, Instant Runoff Voting (IRV), Borda count, approval voting, and STAR-Voting, among others. The audit stops without a full hand count iff all the null hypotheses are rejected. The nulls can be tested in many ways. Ballot polling is particularly simple; two new ballot-polling risk-measuring functions for sampling without replacement are given. Ballot-level comparison audits transform each null into an equivalent assertion that the mean of re-scaled tabulation errors is not greater than 1/2. In turn, that null can then be tested using the same statistical methods used for ballot polling—applied to different finite lists of nonnegative numbers. The SHANGRLA approach thus reduces auditing different social choice functions and different audit methods to the same simple statistical problem. Moreover, SHANGRLA comparison audits are more efficient than previous comparison audits for two reasons: (i) for most social choice functions, the conditions tested are both necessary and sufficient for the reported outcome to be correct, while previous methods tested conditions that were sufficient but not necessary, and (ii) the tests avoid a conservative approximation. The SHANGRLA abstraction simplifies stratified audits, including audits that combine ballot polling with ballot-level comparisons, producing sharper audits than the “SUITE” approach. SHANGRLA works with the “phantoms to evil zombies” strategy to treat missing ballot cards and missing or redacted cast vote records. That also facilitates sampling from “ballot-style manifests,” which can dramatically improve efficiency when the audited contests do not appear on every ballot card. Open-source software implementing SHANGRLA ballot-level comparison audits is available. SHANGRLA was tested in a process pilot audit of an instant-runoff contest in San Francisco, CA, in November, 2019.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See, e.g., https://www.stat.berkeley.edu/users/stark/Vote/auditTools.htm, https://www.stat.berkeley.edu/users/stark/Vote/ballotPollTools.htm, https://github.com/pbstark/auditTools, https://github.com/pbstark/CORLA18/blob/master/code/suite_toolkit.ipynb, and https://github.com/votingworks/arlo (all last visited 10 November 2019); an implementation of SHANGRLA ballot-level comparison audits is available at https://github.com/pbstark/SHANGRLA.
- 2.
A ballot consists of one or more ballot cards. Below, “ballot,” “card,” and “ballot card” are used interchangeably, even though in most U.S. jurisdictions, a ballot consists of more than one ballot card.
- 3.
https://www.github.com/pbstark/SHANGRLA, last visited 22 November 2019.
- 4.
The value might also depend on what the voting system reported for that ballot card and others. See Sect. 3.2.
- 5.
Values \(f \le 1/2\) are not technically “super-majorities,” but the generality is useful. For instance, the rules of some primaries in the U.S. eliminate candidates who receive less than 15% of the vote. An RLA using \(f = 0.15\) might be used to check whether the correct candidates were eliminated.
- 6.
To check whether K candidates all got at least a fraction \(f \in (0, 1)\) of the valid votes (with \(Kf < 1\)) requires testing at most K assertions.
- 7.
- 8.
However, see section Sect. 3.4 below.
References
American Statistical Association: American Statistical Association statement on risk-limiting post-election audits. www.amstat.org/outreach/pdfs/Risk-Limiting_Endorsement.pdf (2010)
Appel, A., deMillo, R., Stark, P.B.: Ballot-marking devices cannot assure the will of the voters. Election Law J. (2020, in press)
Bañuelos, J., Stark, P.: Limiting risk by turning manifest phantoms into evil zombies. Technical report (2012). arXiv.org. http://arxiv.org/abs/1207.3413. Accessed 17 July 2012
Benaloh, J., Jones, D., Lazarus, E., Lindeman, M., Stark, P.: SOBA: secrecy-preserving observable ballot-level audits. In: Proceedings of the 2011 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE 2011), USENIX (2011). http://statistics.berkeley.edu/~stark/Preprints/soba11.pdf
Bernhard, M., et al.: Public evidence from secret ballots. In: Krimmer, R., Volkamer, M., Braun Binder, N., Kersting, N., Pereira, O., Schürmann, C. (eds.) E-Vote-ID 2017. LNCS, vol. 10615, pp. 84–109. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68687-5_6
Bernhard, M., et al.: Can voters detect malicious manipulation of ballot marking devices? In: 41st IEEE Symposium on Security and Privacy. IEEE (2020, in press)
Blom, M., Stuckey, P.J., Teague, V.: Risk-limiting audits for IRV elections. arXiv preprint arXiv:1903.08804 (2019). https://arxiv.org/abs/1903.08804
Feller, W.: An Introduction to Probability Theory and Its Applications, vol. II. Wiley, New York (1971)
Higgins, M., Rivest, R., Stark, P.: Sharper p-values for stratified post-election audits. Stat. Politics Policy 2(1) (2011). http://www.bepress.com/spp/vol2/iss1/7
Kaczmarek, T., et al.: Dispute resolution in accessible voting systems: the design and use of audiotegrity. In: Heather, J., Schneider, S., Teague, V. (eds.) Vote-ID 2013. LNCS, vol. 7985, pp. 127–141. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39185-9_8
Lindeman, M., McBurnett, N., Ottoboni, K., Stark, P.: Next steps for the Colorado risk-limiting audit (CORLA) program. https://arxiv.org/abs/1803.00698 (2018)
Lindeman, M., Stark, P.: A gentle introduction to risk-limiting audits. IEEE Secur. Priv. 10, 42–49 (2012)
Lindeman, M., Stark, P., Yates, V.: BRAVO: Ballot-polling risk-limiting audits to verify outcomes. In: Proceedings of the 2011 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE 2011), USENIX (2012)
National Academies of Sciences, Engineering, and Medicine: Securing the Vote: Protecting American Democracy. The National Academies Press, Washington, September 2018. https://doi.org/10.17226/25120
Ottoboni, K., Bernhard, M., Halderman, J.A., Rivest, R.L., Stark, P.B.: Bernoulli ballot polling: a manifest improvement for risk-limiting audits. In: Bracciali, A., Clark, J., Pintore, F., Rønne, P., Sala, M. (eds.) Financial Cryptography and Data Security, FC 2019. LNCS, vol. 11599. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43725-1_16
Ottoboni, K., Stark, P.B., Lindeman, M., McBurnett, N.: Risk-limiting audits by stratified union-intersection tests of elections (SUITE). In: Krimmer, R., Volkamer, M., Cortier, V., Goré, R., Hapsara, M., Serdült, U., Duenas-Cid, D. (eds.) E-Vote-ID 2018. LNCS, vol. 11143, pp. 174–188. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00419-4_12
Presidential Commission on Election Administration: The American Voting Experience: Report and Recommendations of the Presidential Commission on Election Administration (2014). https://www.eac.gov/assets/1/6/Amer-Voting-Exper-final-draft-01-09-14-508.pdf
Stark, P.: Conservative statistical post-election audits. Ann. Appl. Stat. 2, 550–581 (2008). http://arxiv.org/abs/0807.4005
Stark, P.: A sharper discrepancy measure for post-election audits. Ann. Appl. Stat. 2, 982–985 (2008). http://arxiv.org/abs/0811.1697
Stark, P.: CAST: canvass audits by sampling and testing. IEEE Trans. Inform. Forensics Secur. Spec. Issue Electron. Voting 4, 708–717 (2009)
Stark, P.: Risk-limiting post-election audits: \(P\)-values from common probability inequalities. IEEE Trans. Inf. Forensics Secur. 4, 1005–1014 (2009)
Stark, P.: Super-simple simultaneous single-ballot risk-limiting audits. In: Proceedings of the 2010 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE 2010), USENIX (2010). http://www.usenix.org/events/evtwote10/tech/full_papers/Stark.pdf
Stark, P.: An introduction to risk-limiting audits and evidence-based elections, written testimony prepared for the Little Hoover Commission (2018). https://www.stat.berkeley.edu/~stark/Preprints/lhc18.pdf
Stark, P.B., Teague, V.: Verifiable European elections: risk-limiting audits for D’Hondt and its relatives. JETS: USENIX J. Election Technol. Syst. 3(1) (2014). https://www.usenix.org/jets/issues/0301/stark
Stark, P.B., Wagner, D.A.: Evidence-based elections. IEEE Secur. Priv. 10, 33–41 (2012)
Acknowledgments
I am grateful to Andrew Conway, Steven N. Evans, Kellie Ottoboni, Ronald L. Rivest, Vanessa Teague, Poorvi Vora, and Damjan Vukcevic for helpful conversations and comments on earlier drafts, and to Filip Zagorski for presenting the paper at the 5th Workshop on Advances in Secure Electronic Voting. The SHANGRLA software was a collaborative effort that included Michelle Blom, Andrew Conway, Dan King, Laurent Sandrolini, Peter Stuckey, and Vanessa Teague.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Stark, P.B. (2020). Sets of Half-Average Nulls Generate Risk-Limiting Audits: SHANGRLA. In: Bernhard, M., et al. Financial Cryptography and Data Security. FC 2020. Lecture Notes in Computer Science(), vol 12063. Springer, Cham. https://doi.org/10.1007/978-3-030-54455-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-54455-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-54454-6
Online ISBN: 978-3-030-54455-3
eBook Packages: Computer ScienceComputer Science (R0)