Trustworthiness of voting advice applications in Europe

Voting Advice Applications (VAAs) are interactive tools used to assist in one’s choice of a party or candidate to vote for in an upcoming election. They have the potential to increase citizens’ trust and participation in democratic structures. However, there is no established ground truth for one’s electoral choice, and VAA recommendations depend strongly on architectural and design choices. We assessed several representative European VAAs according to the Ethics Guidelines for Trustworthy AI provided by the European Commission using publicly available information. We found scores to be comparable across VAAs and low in most requirements, with differences reflecting the kind of developing institution. Across VAAs, we identify the need for improvement in (i) transparency regarding the subjectivity of recommendations, (ii) diversity of stakeholder participation, (iii) user-centric documentation of algorithm, and (iv) disclosure of the underlying values and assumptions. Supplementary Information The online version contains supplementary material available at 10.1007/s10676-024-09790-6.

Table A1: Details on case selection and document corpus (a) Elections and document corpus considered in analysis.Open-source code repositories are consulted when available.If application development is outsourced, we additionally consult the information published by the corresponding agency.The listed primary domains include all subsites.The link hosting the respective applications is given in bold.

Kieskompas What2Vote
Kieskompas: Dutch private research agency with a focus on political and societal opinion mapping using quantitative methods founded at the University of Amsterdam which has launched VAAs throughout the world (Kieskompas BV, 2024).

Smartvote
Politools: a politically neutral non-profit scientific network involved in civic education as well as research projects in the field of e-democracy and VAAs.

Wahl-O-Mat
Bundeszentrale für politische Bildung (bpb): German Federal Agency for Civic Education.The digital agency 3pc contributed to web design and development.

SVT Nyheters valkompass
Sveriges Television AB (SVT): Swedish national public broadcaster funded by a public service tax on personal income.

HS Vaalikone
Helsingin Sanomat: a newspaper publisher owned by the media house Sanoma.
Table A2: Common design characteristics and variations as described by Garzia and Marschall (2019).

Content type
a Available in a "rapid" and "deluxe" version.
b Limited to at most 5 items.
a HSTS allows web servers to restrict interaction to HTTPS (Hodges et al, 2012).b DNS CAA reduces the risk of unintended certificate misuse (Hallam-Baker et al, 2019).
c The HttpOnly and Domain flags are optional attributes in the Set − cookie response header.HttpOnly helps mitigate the risk of client side script accessing the protected cookie.If Domain is set too permissively, attacks may be launched on session IDs between different hosts and web applications belonging to the same domain (Open Worldwide Application Security Project (OWASP), 2024).d TLS versions 1.0 and 1.1 were deprecated due to lack of current recommended cryptography algorithms and mechanisms (Moriarty and Farrell, 2021).e At time of analysis the website used Bootstrap 3.3.7 which is vulnerable to cross-site scripting (snyk, 2024).f This column denotes use of Rivest Shamir Adleman algorithm (RSA) for key exchange.As RSA does not ensure Perfect Forward Secrecy, key exposure compromises the secrecy of the complete communication stream (Krawczyk, 2005).g Cipher Block Chaining (CBC) encryption is vulnerable to timing attacks (Al Fardan and Paterson, 2013).h Triple-DES encryption uses the relatively small block size of 64 (Bhargavan and Leurent, 2016).i Secure Hash Algorithm 1 (SHA) is vulnerable to collision attacks (Stevens et al, 2017).
Not fulfilled/no information, (1) implicitly, (2) clear focal point and explicitly given if the VAA is available any language other than the main national onegiven for (i) the definition of values, and (ii) mechanism,

Table A3 :
Design characteristics of the selected VAAs.The ✓ in column Skip indicates that questions can be skipped.Column Weights lists factors that may be applied to questionnaire items, assuming a default weight of 1. Weights > 1 assign higher priority to a question while weights < 1 represent low priority.

Table A4 :
Compliance of each VAA with each sub-requirement.

Table A5 :
Compliance of each VAA over the key requirements in the EGTAI as the mean and standard deviation over compliance across the corresponding sub-requirements.

Table A6 :
VAA support for protocols, cipher suites and standards (R2.5) as well as website configuration (R2.6) related to security.Support for vulnerable technologies is denoted with ×, implementation of security-promoting technologies with ✓.