Skip to main content
SpringerLink
Log in

How to Assess the Usability Metrics of E-Voting Schemes

  • Conference paper
  • First Online:
Financial Cryptography and Data Security (FC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 11599))

Included in the following conference series:

Abstract

Voters play an important role in end-to-end verifiable e-voting schemes because the schemes encourage them to carry out several security-critical tasks by themselves. If the voters cannot complete the tasks by themselves or experience bad usability while executing them, vote manipulations by either a faulty software or deliberate attacks cannot be detected which renders verification useless. Therefore, the scheme’s usability is of crucial importance and demands an early investigation of human factors when implementing e-voting systems. In this paper, we give an overview of user study design challenges when investigating end-to-end verifiable e-voting schemes. We provide guidelines that address these challenges and support researchers in the design of user studies. The guidelines are based on the literature and the authors’ experiences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    We use the subdivision for usability investigation purposes. Note, that the components alone do not replace end-to-end verifiability [14].

  2. 2.

    A scheme for delegation has been proposed [17] whereby the complexity is shifted towards vote casting indicating that even if delegation is possible, human factors are still important.

References

  1. Acemyan, C.Z., Kortum, P., Byrne, M.D., Wallach, D.S.: Usability of voter verifiable, end-to-end voting systems: Baseline data for helios, prêt à voter, and scantegrity ii. USENIX J. Election Technol. Syst. (JETS) 2(3), 26–56 (2014)

    Google Scholar 

  2. Acemyan, C.Z., Kortum, P., Byrne, M.D., Wallach, D.S.: Summative usability assessments of STAR-Vote: a cryptographically secure e2e voting system that has been empirically proven to be easy to use. Hum. factors, 1–24 (2018)

    Google Scholar 

  3. Albert, W., Tullis, T.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Newnes, Boston (2013)

    Google Scholar 

  4. Ali, S.T., Murray, J.: An overview of end-to-end verifiable voting systems. In: Real-World Electronic Voting: Design, Analysis and Deployment, pp. 171–218. CRC Press (2016)

    Google Scholar 

  5. Bederson, B.B., Lee, B., Sherman, R.M., Herrnson, P.S., Niemi, R.G.: Electronic voting system usability issues. In: SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 145–152. ACM (2003)

    Google Scholar 

  6. Benaloh, J., Rivest, R., Ryan, P.Y., Stark, P., Teague, V., Vora, P.: End-to-end verifiability, pp. 1–7 (2015). https://arxiv.org/pdf/1504.03778.pdf

  7. Boren, T., Ramey, J.: Thinking aloud: reconciling theory and practice. IEEE Trans. Prof. Commun. 43(3), 261–278 (2000)

    Article  Google Scholar 

  8. Brooke, J.: SUS - a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  9. Budurushi, J., Renaud, K., Volkamer, M., Woide, M.: An investigation into the usability of electronic voting systems for complex elections. Ann. Telecommun. 71(7), 309–322 (2016). https://doi.org/10.1007/s12243-016-0510-2

    Article  Google Scholar 

  10. Byrne, M.D., Greene, K.K., Everett, S.P.: Usability of voting systems: baseline data for paper, punch cards, and lever machines. In: SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 171–180. ACM (2007)

    Google Scholar 

  11. Campbell, B.A., Byrne, M.D.: Now do voters notice review screen anomalies? A look at voting system usability. In: Conference on Electronic Voting Technology/Workshop on Trustworthy Elections (EVT/WOTE). USENIX Association (2009)

    Google Scholar 

  12. Chaum, D.: Secret-ballot receipts: true voter-verifiable elections. IEEE Secur. Priv. 2(1), 38–47 (2004)

    Article  Google Scholar 

  13. Conrad, F.G., et al.: Electronic voting eliminates hanging chads but introduces new usability challenges. Int. J. Hum.-Comput. Stud. 67(1), 111–124 (2009)

    Article  Google Scholar 

  14. Cortier, V., Galindo, D., Küsters, R., Mueller, J., Truderung, T.: SoK: verifiability notions for e-voting protocols. In: Symposium on Security and Privacy (S&P), pp. 779–798. IEEE (2016)

    Google Scholar 

  15. Culnane, C., Teague, V.: Strategies for voter-initiated election audits. In: Zhu, Q., Alpcan, T., Panaousis, E., Tambe, M., Casey, W. (eds.) GameSec 2016. LNCS, vol. 9996, pp. 235–247. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47413-7_14

    Chapter  MATH  Google Scholar 

  16. Distler, V., Zollinger, M.L., Lallemand, C., Rønne, P.B., Ryan, P.Y., Koenig, V.: Security-visible, yet unseen? How displaying security mechanisms impacts user experience and perceived security. In: CHI Conference on Human Factors in Computing Systems, CHI 2019. ACM (2019)

    Google Scholar 

  17. Escala, A., Guasch, S., Herranz, J., Morillo, P.: Universal cast-as-intended verifiability. In: Clark, J., Meiklejohn, S., Ryan, P.Y.A., Wallach, D., Brenner, M., Rohloff, K. (eds.) FC 2016. LNCS, vol. 9604, pp. 233–250. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-53357-4_16

    Chapter  Google Scholar 

  18. Everett, S.P., Byrne, M.D., Greene, K.K.: Measuring the usability of paper ballots: efficiency, effectiveness, and satisfaction. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50, no. 24, pp. 2547–2551 (2006)

    Google Scholar 

  19. Everett, S.P., et al.: Electronic voting machines versus traditional methods: improved preference, similar performance. In: SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 883–892. ACM (2008)

    Google Scholar 

  20. Fuglerud, K.S., Røssvoll, T.H.: An evaluation of web-based voting usability and accessibility. Univ. Access Inf. Soc. 11(4), 359–373 (2012). https://doi.org/10.1007/s10209-011-0253-9

    Article  Google Scholar 

  21. Gibson, J.P., MacNamara, D., Oakley, K.: Just like paper and the 3-colour protocol: a voting interface requirements engineering case study. In: International Workshop on Requirements Engineering for Electronic Voting Systems, pp. 66–75. IEEE (2011)

    Google Scholar 

  22. Gjøsteen, K.: The Norwegian internet voting protocol. In: Kiayias, A., Lipmaa, H. (eds.) Vote-ID 2011. LNCS, vol. 7187, pp. 1–18. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32747-6_1

    Chapter  Google Scholar 

  23. Greene, K.K., Byrne, M.D., Everett, S.P.: A comparison of usability between voting methods. In: Electronic Voting Technology Workshop (EVT). USENIX Association (2006)

    Google Scholar 

  24. Grimm, P.: Social desirability bias. Wiley International Encyclopedia of Marketing. Wiley, Hoboken (2010)

    Book  Google Scholar 

  25. Herrnson, P.S., Niemi, R.G., Hanmer, M.J., Bederson, B.B., Conrad, F.G., Traugott, M.: The importance of usability testing of voting systems. In: Electronic Voting Technology Workshop (EVT) (2006)

    Google Scholar 

  26. de Jong, M., van Hoof, J., Gosselt, J.: User research of a voting machine: Preliminary findings and experiences. J. Usability Stud. 2(4), 180–189 (2007)

    Google Scholar 

  27. Karayumak, F., Kauer, M., Olembo, M.M., Volk, T., Volkamer, M.: User study of the improved Helios voting system interfaces. In: Workshop on Socio-Technical Aspects in Security and Trust (STAST), pp. 37–44. IEEE (2011)

    Google Scholar 

  28. Kulyk, O., Neumann, S., Budurushi, J., Volkamer, M.: Nothing comes for free: How much usability can you sacrifice for security? IEEE Secur. Priv. 15(3), 24–29 (2017)

    Article  Google Scholar 

  29. Laskowski, S.J., Autry, M., Cugini, J., Killam, W., Yen, J.: Improving the usability and accessibility of voting systems and products. NIST Spec. Publ. 500, 256 (2004)

    Google Scholar 

  30. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 63–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89350-9_6

    Chapter  Google Scholar 

  31. Mac Namara, D., Scully, T., Gibson, P.: Dualvote addressing usability and verifiability issues in electronic voting systems (2011). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.399.7284

  32. MacNamara, D., Carmody, F., Scully, T., Oakley, K., Quane, E., Gibson, J.P.: Dual vote: a novel user interface for e-voting systems. In: International Conference on Interfaces and Human Computer Interaction, pp. 129–138. IADIS (2010)

    Google Scholar 

  33. MacNamara, D., Gibson, P., Oakley, K.: A preliminary study on a dualvote and Prêt à Voter hybrid system. In: Conference for E-Democracy and Open Government, p. 77. Edition-Donau-Univ. Krems (2012)

    Google Scholar 

  34. Marky, K., Kulyk, O., Renaud, K., Volkamer, M.: What did I really vote for? On the usability of verifiable e-voting schemes. In: CHI Conference on Human Factors in Computing Systems (CHI), pp. 176:1–176:13. ACM (2018)

    Google Scholar 

  35. Marky, K., Kulyk, O., Volkamer, M.: Comparative usability evaluation of cast-as-intended verification approaches in internet voting. In: SICHERHEIT 2018, pp. 197–208. Lecture Notes in Informatics (LNI), Gesellschaft für Informatik (2018)

    Google Scholar 

  36. Marky, K., Schmitz, M., Lange, F., Mühlhäuser, M.: Usability of code voting modalities. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA 2019, pp. LBW2221:1–LBW2221:6. ACM (2019)

    Google Scholar 

  37. Nancarrow, C., Brace, I.: Saying the “right thing”: coping with social desirability bias in marketing research. Bristol Bus. Sch. Teach. Res. Rev. 3(11), 1–11 (2000)

    Google Scholar 

  38. Neumann, S.: Evaluation and improvement of internet voting schemes based on legally-founded security requirements. Ph.D. thesis, Technische Universität Darmstadt (2016)

    Google Scholar 

  39. Olembo, M.M., Volkamer, M.: E-voting system usability: Lessons for interface design, user studies, and usability criteria. In: Human-Centered System Design for Electronic Governance, pp. 172–201. IGI Global (2013)

    Google Scholar 

  40. Patrick, A.: Ecological validity in studies of security and human behaviour. In: Symposium on Usable Privacy and Security (SOUPS) (2009)

    Google Scholar 

  41. Realpe-Muñoz, P., Collazos, C.A., Hurtado, J., Granollers, T., Muñoz-Arteaga, J., Velasco-Medina, J.: Eye tracking-based behavioral study of users using e-voting systems. Comput. Stan. Interfaces 55, 182–195 (2017)

    Article  Google Scholar 

  42. Selker, T., Rosenzweig, E., Pandolfo, A.: A methodology for testing voting systems. J. Usability Stud. 2(1), 7–21 (2006)

    Google Scholar 

  43. Sherman, A.T., et al.: Scantegrity mock election at Takoma Park. In: International Conference on Electronic Voting (EVOTE), pp. 45–61. LNI, Gesellschaft für Informatik (2010)

    Google Scholar 

  44. Standardization, I.O.F.: ISO 9241–11: Ergonomics of human system interaction - part 11: Guidance on usability (1998)

    Google Scholar 

  45. Standardization, I.O.F.: ISO 9241–210: Part 210: Human-centred design for interactive systems (2015)

    Google Scholar 

  46. Strafgesetzbuch (StGB): §107c Verletzung des Wahlgeheimnisses. https://www.gesetze-im-internet.de/stgb/__107c.html

  47. Van Hoof, J.J., Gosselt, J.F., de Jong, M.D.: The reliability and usability of the NEDAP voting machine: a pilot study. University of Twente Faculty of Behavioural Sciences Department of Technical and Professional Communication (2007)

    Google Scholar 

  48. Weber, J.L., Hengartner, U.: Usability study of the open audit voting system Helios. http://www.jannaweber.com/wp-content/uploads/2009/09/858Helios.pdf (2009)

  49. Winckler, M., et al.: Assessing the usability of open verifiable e-voting systems: a trial with the system prêt à voter. In: International Conference on Theory and Practice of Electronic Governance (ICEGOV), pp. 281–296. ACM (2009)

    Google Scholar 

Download references

Acknowledgements

This work has been co-funded by the DFG as part of project “Area D.1” within the RTG 2050 “Privacy and Trust for Mobile Users” and by the Horst Görtz Foundation. We acknowledge support from the Luxembourg National Research Fund (FNR) for funding, in particular Marie-Laure Zollinger was supported by the FNR-INTER-VoteVerif project and FNR-INTER-SeVoTe project.

Author information

Authors and Affiliations

  1. Telecooperation Lab, Technische Universität Darmstadt, Darmstadt, Germany

    Karola Marky, Markus Funk & Max Mühlhäuser

  2. University of Luxembourg, Luxembourg City, Luxembourg

    Marie-Laure Zollinger & Peter Y. A. Ryan

Authors
  1. Karola Marky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marie-Laure Zollinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Markus Funk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Y. A. Ryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Max Mühlhäuser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Karola Marky .

Editor information

Editors and Affiliations

  1. Stirling University, Stirling, UK

    Andrea Bracciali

  2. Concordia University, Montréal, QC, Canada

    Jeremy Clark

  3. University of Oxford, Oxford, UK

    Federico Pintore

  4. University of Luxembourg, Esch-sur-Alzette, Luxembourg

    Peter B. Rønne

  5. University of Trento, Trento, Italy

    Massimiliano Sala

Rights and permissions

Reprints and permissions

Copyright information

© 2020 International Financial Cryptography Association

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Marky, K., Zollinger, ML., Funk, M., Ryan, P.Y.A., Mühlhäuser, M. (2020). How to Assess the Usability Metrics of E-Voting Schemes. In: Bracciali, A., Clark, J., Pintore, F., Rønne, P., Sala, M. (eds) Financial Cryptography and Data Security. FC 2019. Lecture Notes in Computer Science(), vol 11599. Springer, Cham. https://doi.org/10.1007/978-3-030-43725-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-43725-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-43724-4

  • Online ISBN: 978-3-030-43725-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation