1 Introduction

I first met Cédric when we were high-school students. We had a few classes in common and played sports together. I particularly recall our football matches. Cédric was better than I at that discipline. We continued our studies at university and found ourselves students in business informatics. I was not a hardworking student. I was sitting at the back of the class, while Cédric was in the front row. I must say that Cédric is visually impaired. Because of a medical error during his preterm birth, he lost vision in one eye, and the second eye's vision is only 10%. However, he does not consider himself disabled. He has completed his university studies and has worked for various companies. Nevertheless, in this digital society, Cédric has lost his autonomy. Therefore, his differences are more noticeable in the eyes of others. Since Cédric is a member of our research team, we adopted his perspective to study this societal issue.

Until recently, Cédric could buy a train ticket at a physical counter with the train station worker remaining unaware of his disability. Today, Cédric is referred to a digital kiosk or a smartphone app, forcing him to confess his disability so that a worker will help him. Technologies seem to build new forms of barriers that exclude some individuals [1]. One speaks of digital exclusion, digital divide, or digital inequality to describe the exclusion of technologies in the lives of individuals [2]. To limit the digital divide, scientific research has been interested in information accessibility practices over the last fifteen years [3].

Of course, Cédric is not an isolated case. According to the World Health Organization, there are about 2.2 billion people with visual impairments (PVI) around the world [4]. PVI includes sub-types of visual impairment and takes into account people with blindness as well as people with low vision [5]. Technological advances provide them opportunities to improve their living conditions, for instance, by using assistive technologies. While people with disabilities enjoy a higher level of digital inclusion today than in the past, they are still at great risk of digital exclusion [6]. The elimination of human-operated service counters has created service exclusions for PVI. Thereby, access to information and digital services is a fundamental right [7]. In many countries, acts require that services be made available to all populations and demonstrate the major interest of our concern (e.g., the UK Equality Act 2010 or the Americans with Disabilities Act). In some countries, such as Switzerland, the Act (LHand) and Ordinance (OHand) specify that the information and digital services offered on the Internet must be accessible to people with speech, hearing, vision or motor disabilities. To this end, websites and digital services must be designed in accordance with international technical standards, in particular the guidelines for the accessibility of web pages issued by the World Wide Web Consortium (W3C) as well as national technical standards. Nevertheless, these guidelines are not sufficient to guarantee accessibility of web pages [8]. Moreover, there is no consensus on the very notion of accessibility in either scientific literature or in international standards [9].

In a period marked by the digital transformation of society and its related risks [10], it seems appropriate to look at a certain dark side of the latter. While prior research has attempted to examine disability as a cause of digital divide, this present research work identified two successive research questions: first, to what extent do technical standards support PVI to overcome the digital divide; second, to what extent the digital information of an electronic state voting system is understood by PVI. To answer these research questions, we conducted an action research in the context of an electronic state voting system in Switzerland. Indeed, the Swiss political system allows citizens to vote on the decisions of the federal parliament or even to propose changes to the constitution. Swiss citizens have the right to vote and are called on to exercise this right, on average, on 15 federal proposals per year. Voting systems must allow all citizens to vote without excluding a minority.

The next section proposes a literature review to clarify the concept of digital divide, and a potential response promoted by web accessibility. Next, we present our detailed action research methodology and provide the context in which our action research was conducted. We then detail the three cycles of our action research. During the first cycle, we address accessibility for PVI via compliance with technical standards. During the second, we widen our considerations beyond technical standards. The third cycle is confirmatory. Finally, we recall the contributions of this research, highlight its limitations and propose future research paths.

2 Background

To answer to what extent do technical standards support PVI to overcome the digital divide, it seems necessary to clarify the concept of digital divide, its causes and remedies, with a focus on disabilities. To best highlight the seminal publications related to this concept, we combined an interpretive literature review with a bibliometric analysis as suggested by Walsh and Renaud [11]. This approach can be illustrated by the flesh and bones metaphor, where researchers' interpretation of documents is put on a bibliometric analysis which reveals the structure of the field of research. More specifically, we conducted three Reference Co-Citation Analyses (RCCA). A bibliometric co-citation analysis differs from a more traditional structured literature review in terms of the use of statistical techniques on a large number of references (i.e., those cited by documents).

Several steps are necessary to conduct a literature review using RCCA. First, it consists of identifying a set of documents (e.g., articles) that respond to our problem in a scientific database (e.g., Scopus, Web of Science). These documents constitute the first-order sample. Second, the set of references cited by the first-order sample is extracted and processed by statistical techniques to constitute a second-order sample. Two references are co-cited when they are cited simultaneously in a document of the first-order sample [11]. The hypothesis behind RCCA is that the more two units are co-cited, the more related they are within the same school of thought (sometimes supporting, sometimes contradicting) [11]. Fourth, the interpretative analysis of the literature is performed on the most co-cited references. RCCA is considered relevant for conducting literature reviews and to identify seminal works of a research subfield [11, 12]. Its main objective is to identify publication patterns, to classify published research, in order to show the intellectual tradition or network of a field. Co-citation analyses are most used to elucidate the intellectual structure of disciplines [12].

We used the software ARTIREV [13] to support our bibliometric co-citation analyses. ARTIREV offers an extraction interface for scientific documents indexed by Scopus. Concretely, we have constituted three first-order samples by extracting scientific publications related to (1) digital divide, (2) web accessibility with a focus on disabilities and (3) web accessibility in the context of e-government (see appendixes A to D for further details). Scopus provides the necessary bibliographic records (references) for applying reference-based processing. We therefore obtained all the references cited in the first-order samples to constitute the second-order samples. To ensure the reliability of the results, we performed semiautomatic data cleaning of the second-order samples aided by ARTIREV's bibliographic reference cleaning functions. The cleaning phase is essential because the references cited in the articles often contain errors (typos or incorrect citations). We also performed multiple analyses to select the most appropriate statistical threshold. Finally, we interpreted our results using a VOS distance-based map [14]. The literature review presented in this paper is based on an in-depth interpretive analysis of clusters that emerged from the RCCAs. Readers who want more information on bibliometric analysis methods may refer to the Zupic and Ĉater paper [15].

Regarding our interpretative analysis, we first present a narrative based on commonalities between works. Secondly, we particularly focused on the perspective of the work (e.g., accessibility and/or usability), the community of focus (e.g., PVI, people with disabilities) and the theory type [16]. Gregor [16] provides a taxonomy that classifies information system (IS) theories with respect to their goal: (1) theory for analyzing, (2) theory for explaining, (3) theory for predicting, (4) theory for explaining and predicting and (5) theory for design and action. Since not all scientific works aim to create a theory, but provide some practical advice, we have added a category that characterized practical contribution (i.e., none, few, several).

2.1 Digital divide and disability

Regarding “digital divide and disability,” the foundations of the research field are organized into three clusters (Appendix B). The first cluster, composed of books that are empirically rooted, presents the foundations of digital divide. The digital divide is presented as a multifaceted concept and a complex phenomenon, while the disability is seen as one cause. The second cluster, composed of seven publications, concerns empirical investigations on digital divide caused by disability. These investigations are focused on the access and use of information and communication technologies (ICT) and present the main explanatory variables of digital divide. The third cluster contains three publications regarding one kind of disability and digital divide.

2.1.1 Digital divide

The concept of digital divide gained momentum in the mid-1990s with the adoption of the Internet [17]. In a first understanding, the digital divide is a boundary that separates individuals into two worlds: the connected world and the world of the left-behind who cannot be connected [17, 18]. The connection relates to physical access to digital devices and an Internet connection [18]. Computers and the Internet are considered as new digital media because they are digital, converging and networked [17,18,19]. The digital divide has opened a broad debate on whether individuals have sufficient access to information and ICT. However, a tripartite distribution of the society (information elite, digitally illiterate and the majority of the population between these extreme) conducts to a better distinction than a two-tiered society conveyed by the divide metaphor [18]. Authors argue that the digital divide is a polymorphous and thus a complex concept. For instance, Norris [20] analyzed the digital divide as a multidimensional phenomenon encompassing three distinct forms of divides: global, social and democratic. Warschauer [17] studied this phenomenon with a focus on the social dimension. Moreover, authors distinguish two levels of digital divide. The first level separates people with access to ICT from those without, while the second level is focused on the differences in what people do online once they have gained access. The development of the field followed the same logic. First empirical investigations were particularly interested in the gaps of possession, while subsequent works analyzed both Internet access and use with a focus on the usage gap [21]. Internet accessibility not only depends on technical aspects, but also on disability [21], and several other factors, such as education, economic status and geographic residence [22].

2.1.2 Disability as a cause of digital divide

The “digital disability divide” or “disability digital divide” concerns the digital divide among people with disabilities in the context of ICT [21]. Authors agree that digital divide can be caused by or deepened by the disability which impacts both access and use of ICT [19, 23, 24].

The digital disability divide has been explored regarding Internet access and use from both micro- and macroperspectives. An individual model perspective with a microfocus does not explain entirely the phenomenon [1]. On the other hand, there is some evidence that macro or structural barriers prevent people with disabilities from using digital technologies [1]. Disability status is often associated with lower levels of income and employment, and causes considerably lower levels of ICT use [25]. Even after controlling for demographic and socioeconomic factors, people with disabilities still trail those without disabilities with regard to Internet access, adoption and use [21]. In addition, the access gap can be explained by the time needed to develop accessible technologies [1]. Guo et al. [22] adopted the social model of disability that focuses on environmental barriers rather than the impairment of individuals. They reported that economic status, geographic residence and sources of Internet-related knowledge were the most influential social factors associated with Internet accessibility.

Finally, ICT plays an important role to support people with intellectual disabilities in their usage of the Internet and particularly social media [26, 27]. Therefore, the type of disability and impairment-related factors impedes access and use of ICT. Chadwick et al. [28] demonstrated that functional variations and impairments (the education, training and support, and wider political, economic and attitudinal climate) are combined in complex and varied ways to marginalize people with intellectual disabilities from accessing and using the Internet.

In summary, disability combined in complex and varied ways with structural factors impacts both ICT access and use by people with disabilities.

2.2 Visual impairment and web accessibility

The subfield of “Visual impairment and Accessibility” highlights two clusters (cf. Appendix C). The first cluster contains publications interested in web accessibility guidelines and standards, compliance and empirical investigations on this topic. The second cluster contains empirical investigations on visual impairment and web usage. It seems important to highlight that empirical investigations have studied this phenomenon in both accessibility and the human–computer interaction (HCI) fields. Also, the RCCA highlights different populations: those who design web content, those who use web content (people with disabilities) and those who establish guidelines and standards. The accessibility guidelines between these populations concern and impact differently both of them.

2.2.1 Accessibility of ICT as a remedy of the disability digital divide

Accessibility standards and guidelines serve as a remedy of the disability digital divide. The first and the second versions of the Web Content Accessibility Guidelines (WCAG) are inevitably the most cited documents [29, 30]. The WCAG aims to make content accessible to a wider range of people with disabilities by providing a single shared standard that meets the needs of individuals, organizations and governments internationally. Paciello [31] and Thatcher et al. [32] performed a complete overview of technical, legal and practical aspects of web accessibility. Elsewhere, Gibson [33] provides an overview of accessibility capabilities of the Web 2.0 by reporting a representative toolbox.

Regarding empirical investigations, researchers first analyzed the conformance of common websites to accessibility guidelines [34]. Due to the lack of respect of accessibility guidelines and standards [34, 35], the reasons have been formalized and advice on their integration has been provided. The lack of time, training, managerial and client support, inadequate software tools, as well as confusing accessibility guidelines are the main barriers to accessibility [35]. Elsewhere, accessibility only constraints the visual design (i.e., layout and appearance) of websites when the creative freedom is the main criterion of success [36]. To overcome these limitations, authors investigated how to integrate accessibility standards into the software development process [35, 37]. Lazar et al. [35] developed the Web Accessibility Integration Model which highlights multiple points within web development where accessibility can be incorporated or forgotten. Moreover, website developers must have lightweight evaluation techniques that support an iterative design [38]. Lightweight techniques such as automated tools can be combined with heavy laboratory experiments.

2.2.2 Web accessibility for people with visual impairments

Regarding visual impairment and web accessibility, most authors investigated the behaviors of PVI users on the web, the problems they encounter and the use of assistive technologies [39, 40].

Bigham et al. [41] compared the differences in the browsing behavior of blind and sighted web users by recording the visited pages and actions on them. Blind users avoided pages containing severe accessibility problems, while the dynamic content characteristic of the Web 2.0 deters web accessibility. The main causes of frustration experienced by blind users using screen readers are related to web page content and structure and technical issues (e.g., conflict between screen reader and applications) [39]. To cope with accessibility or usability issues, PVI users mobilized adaptive strategies. As a consequence, accessibility problems may not be as important as the ones that do not have adaptive strategies [40]. Blind users sometimes surprisingly used the mouse cursor when page elements were otherwise inaccessible [41]. Also, PVI can combine multiple adaptive strategies in one situation, such as zooming with a screen magnifier and listening to the text with a screen reader.

Finally, some authors developed solutions to answer the problems encountered by PVI, such as an accessible Internet browser to control media suitably integrated with screen readers [42], or a non-visual user interface [43]. Some tools help developers in the design of accessible web pages [44, 45].

2.2.3 Evaluation beyond accessibility guidelines

Authors agree with the idea that respecting accessibility regulations alone is not sufficient [34, 37, 41, 46]. As mentioned by Theofanos and Redish [37], guidelines, based on technology, are not enough to meet the user needs. Thus, it is necessary to understand the users and how they work with their tools. Sullivan and Matson [34] distinguish the accessibility from the usability or the inaccessible from the unusable by discussing the dichotomous nature of accessibility evaluations. In the context of visual impairment, compliant web pages are often not truly usable by blind users [45]. This situation can be explained because evaluation tools were merely focused on technical verification such as HTML tags with respect to regulations and guidelines. Moreover, WCAG, which aims to answer all disabilities, is generic by nature, while vision impairments refer to one kind of disability implying diverse sub-populations. Furthermore, Power et al. [46] divided the overall set of user problems into three types: (1) problems not covered by guidelines, (2) those covered by guidelines but the guidelines are not implemented and (3) those covered by guidelines with guideline implementations. They highlighted that the WCAG 2.0 covered about 50% of problems encountered by users and did not improve the usability. They recommend web accessibility research to be inspired by usability research, for instance, by defining a much broader set of design principles based on user data and the use of the web by people with disabilities. In this perspective, usability is the second half of the accessibility story. Following that way of thinking, accessibility could be seen as a prerequisite to usability.

2.3 Accessibility in e-government

The subfield of “Accessibility in e-government” highlights two clusters (cf. Appendix D). The first cluster, composed of eleven publications, addresses the empirical evaluation of web accessibility in the e-government context. Not surprisingly, we found a second cluster with similar content to that of the first cluster of the RCCA analysis concerning “Visual impairment and accessibility” (i.e., web accessibility norms).

2.3.1 Web accessibility evaluation in the e-government context

We observe that studies of our set evaluated the accessibility of e-government at different levels. Four studies were focused on one or multiples states [47,48,49], seven performed an analysis at a national level [50,51,52,53,54,55], and one took interest at an international level [56]. All evaluations reveal that government websites had limited accessibility for people with disabilities, although the law requires it to be taken into account (e.g., [49, 54]). Authors argue that accessibility unawareness [49, 50], limited adoption [55], limited resources (e.g., financial, human) [49, 52, 55, 56], and limited methods [55] and tools [47] are the main reason to explain such results. Studies suggest addressing the developers of e-government websites themselves, for instance by considering accessibility during site design [53, 54].

In order to evaluate e-government sites, researchers relied on the W3C guidelines and adopted a norm or standard-oriented perspective. In this approach, evaluations are mostly performed through automatic validation which are based on WCAG 1.0 or 2.0 as well as national legislation (e.g., Sect. 508). Relying on automatic evaluations is highlighted by the importance of the WAVE evaluation tool (cf. Appendix D). Human verifications are rarely performed in combination with an automatic evaluation [55]. Moreover, the presence of accessibility laws or guidelines does not guarantee that e-government sites are accessible [56]. That is why researchers addressed a wider view of accessibility. We emphasize Jaeger's [53] more holistic view to evaluate US e-government websites that considered policy, users, experts and developers, as well as Ismailova's perspective considering accessibility, usability and security to evaluate websites of the Kyrgyz Republic [52].

2.4 Interpretative analysis related to web accessibility

An overview of our interpretative analysis of journal articles or conference proceedings related to web accessibility is provided in Table 1. Documents are organized in regard to their focus within web accessibility (i.e., corresponding to the subfield of research) and ordered by the name of the first author.

Table 1 Interpretative analysis of scientific publications related to web accessibility in our second-order samples in terms of community of focus (PWD people with disabilities, blind, specialist such as designer, or developer, perspective (accessibility, usability, other), theory type in IS (1-Analysis, 2-Explanation, 3-Prediction, 4-Explanation and prediction, 5-Design and action, or '–' when theory creation is not the aim of the paper) and practical advice ('–' or none, few or several)
Full size table

As previously stated, researchers agree that compliance with accessibility rules is not sufficient. Thus, research in the area of e-government accessibility must consider accessibility as a prerequisite but not a sufficient condition for the adequate use of e-government services for people with disabilities. Our literature review confirms the relevance of our initial research question, namely to what extent do technical standards support PVI to overcome the digital divide. This question is related to the implementation of accessibility through technical standards. Despite the increasing interest in accessibility guidelines, methods, tools and processes within software engineering [57], web accessibility evaluations in an e-government context are mainly based on automatic checking tools, and developers faced several difficulties to apply accessibility principles. Action research (AR) seems well-suited to address these issues and complements the current body of knowledge. AR has been used in social sciences since 1940 [58] and can take many forms [59]. For Baskerville and Myers, “action research aims to solve current practical problems while expending scientific knowledge” [60]. AR increases the understanding of urgent social problems and is very applicable to the IS domain; simultaneously assists pragmatic problem solving while expanding knowledge; collaboratively enhances the competencies of the respective actors; and applies well to the understanding of social change processes [59, 61]. In AR, researchers observe and participate in the phenomena studied [61]. It should be noted that AR was originally based on a two-stage process: the diagnostic stage and the therapeutic stage. The diagnostic stage of the process involves a collaborative analysis of the research participants. The therapeutic stage of the process entails collaborative change experiments in the research field [61]. AR can be implemented through a cyclical process in five phases: diagnosis, action planning, action taking, evaluating and specifying learning. The joint activities carried out by the client and the action researcher maintain and regulate all or part of these five phases [62].

3 Method

3.1 Research context

Switzerland is a federal state in which power is divided between the Confederation (national), the cantons (regional) and the communes (local) [63]. This separation of powers allows cantons to make decisions regarding their taxation, their budgets, their education, and their political system. Direct democracy is implied. Thus, Swiss citizens represent the highest authority in the land and directly participate in political decisions. As well as the right to vote in elections and referendums, Swiss citizens may voice their demands by means of three instruments that form the core of direct democracy: popular initiative, mandatory referendum and optional referendum [64]. The population votes and elects the executive council in three ways: traditionally via ballot boxes; by mail (most common); and by electronic voting. The third type is the focus of our research.

Electronic voting is one of the e-government IS services available to the population of the canton in which we conducted our research. Available as a web virtual counter, more than 100 services (e.g., online declaration and tax file, school supervision of children and compulsory insurance or mobility) enable individuals and other legal entities to interact with the Administration. To comply with Swiss ordinances, such virtual counters must be accessible to people with disabilities. Hence, accessibility is defined as "the extent to which products, systems, services, environments and facilities are able to be used by a population with the widest range of characteristics and capabilities, to achieve a specified goal in a specified context" [9]. Respecting accessibility principles is particularly important for electronic voting, which must comply in our context with the Swiss accessibility standard (eCH-0059) or face withdrawal of accreditation. We point out that the version 2.0 of the eCH-0059 is aligned with the WCAG 2.0. This standard is to be applied primarily to all state or public institutions’ websites, such as the Confederation, cantons, municipalities, universities, schools, hospitals, libraries, particularly in the area of e-government and e-voting. Both existing and new websites must meet the AA compliance level.

3.2 Action research procedure

To answer our initial research question, we conducted an AR in the context of an electronic state voting system in Switzerland. This methodology suits the interventionist aim of our work as well as our posture. In our case, we address a real need for change (i.e., the legal aspect of technical accessibility), seek to solve problems iteratively, optimize collaboration with practitioners and create knowledge that is theoretically grounded and relevant.

We used the AR approach of Susman and Evered [62], as adapted by Baskerville and Pries–Heje [65]. Thus, after negotiating our action with a research field, we conducted three AR cycles to solve current practical problems while answering our research question. Our first cycle ran from August 2017 to March 2018; the second ran from March 2018 to January 2019; the third ran in February 2021. We established a research environment infrastructure [59], and, based on agile methods, clarified the aim of our first cycle in collaboration with our research partners. The aim was to evaluate and optimize the technical accessibility of the electronic voting IS to PVI by respecting Swiss and international standards. Therefore, our first cycle focuses on a practical technical problem solving [66] (i.e., in order to answer our research question). As proposed by Baskerville and Pries–Heje [65], our cycles comprised five phases: (1) diagnosis, (2) action plan, (3) action, (4) evaluation and (5) learning. Because of the cyclic nature of IS design, we integrated iterations between the action and evaluation phases. This agile practice seems to be rooted in the practices of designing artifacts in IS. At the end of the first cycle, a second research question emerged from our research work. Hence, we focused on this new research question that acted as the entry point for a second cycle. Thus, and as proposed by McKay and Marshall [66], our second cycle focuses on a research problem, which distances us from consultancy. This activity is also consistent with the AR approach, which proposes to complete multiple cycles [67]. At the end of the second cycle, we conducted a third confirmatory cycle. Figure 1 summarizes our methodological approach.

Fig. 1
figure 1

Synthesis of the cycles of our action research

Full size image

4 Results and discussion

4.1 First cycle: technical accessibility of electronic voting IS through standards

4.1.1 Research environment

Our action was initiated by the demand of a Swiss canton that needed to obtain federal accreditation in order to be allowed to open electronic voting before the next votes on 4 March 2018. In August 2017, we negotiated our research objectives, scope, costs and deadlines. We agreed that the research objectives would be established collaboratively at the beginning of each AR cycle. We further clarified the role of researcher, which is not synonymous with consultant. Lastly, we agreed that the results and knowledge acquired in this action research would be published.

Next, we formed the action research team, leveraging one researcher with severe visual impairments (Cédric). Three information technology (IT) practitioners from the canton became co-participants, in accordance with action research practices [61, 68]. We agreed that the composition of the action research team could be adapted as we move forward. To initiate this AR cycle, we have negotiated our first aim: evaluating and optimizing electronic voting technical accessibility for PVI in compliance with Swiss and international standards.

4.1.2 Diagnosis

Diagnosis consisted of three activities. First, we conducted a participatory workshop with the full team plus two frontend developers of the electronic voting IS. Everyone presented their knowledge about the accessibility of electronic voting, specifically regarding capabilities for PVI. The frontend developers then presented technical elements in this regard. Relevance is obtained during the research in the company, and rigor when taking a step back in the "laboratory." Hence, to alternate research in companies and in "laboratories" outside the company, we simultaneously reviewed extant literature on these subject and related legal documents. Thus, we built a body of knowledge that converged on the scientific literature and practical sources. We then performed an initial assessment of an electronic voting IS mock-up. Our objective was to determine whether certification would be possible. After having evaluated this technological artifact, we confirmed that reaching the certification was possible. However, the mock-up did not achieve the AA compliance level required by the Swiss standard (eCH-0059 2.0) to obtain certification. Finally, in October 2017, we sent to the canton a report describing our activities, presenting our initial diagnosis and providing technical advice to ensure that the electronic voting IS complies with the Swiss accessibility standard at the AA compliance level.

4.1.3 Action plan

Following our diagnosis, we planned activities to address the issue of Internet access to the electronic voting IS for PVI. In order to identify them, we organized meetings with organizers and developers. We used the approach of Baskerville and Pries–Heje [65], who, for problem-solving strategies, recommended that participants specify the target object of change followed by an approach for achieving it. The accessibility of the virtual counter was the target of our desired change. Our collaboration involved defining the testing and coding requirements for the improved electronic voting capabilities, noting that accreditation must be performed by an independent department of the canton, competent according to OVotE, Ch. 2.1, and recognized by the Federal Chancellery. The following activities resulted (points 3 to 5 are iterative):

  1. (1)

    Obtain recognition by the Federal Chancellery for researchers according to OVotE.

  2. (2)

    Define the scope of the accessibility evaluation.

  3. (3)

    Evaluate the technical accessibility.

  4. (4)

    Provide advice to adapt the source code of the electronic voting IS.

  5. (5)

    Change the source code to meet technical accessibility requirements.

  6. (6)

    Write a report and a formal letter of accessibility compliance according to eCH-0059 2.0.

  7. (7)

    Submit the electronic voting IS to a follow-up test at the federal level to obtain accreditation.

4.1.4 Action

The action research team cooperated to complete the seven activities. To obtain recognition according to OVotE, we obtained a written recognition from the Federal Chancellery in September 2017, demonstrating that we are able to evaluate all points of the eCH-0059 standard. We then defined the scope of action to be performed on the electronic voting IS system.

We had to verify the AA conformity level of 15 web pages with authentication, navigation, electronic voting services, acknowledgment of receipt-control services and PDF documents presenting voting objects. To do so, we relied on the checklist associated with eCH-0059 2.0 (Accessibility Checklist 2.0) which contains 105 checkpoints. In total, more than 2,000 manual verifications have been performed via an expert evaluation of two members of the research team, one with severe visual impairments (Cédric). Moreover, both evaluators relied also on automatic checking tools recommended by the W3C such as the HTML Markup Validation Service (validator.w3.org) and a PDF accessibility checker. The PDF accessibility checker version 3 is part of the helper tools provided by the Swiss Access for All Foundation (https://www.accessibility-developer-guide.com/setup/helper-tools).

Again, electronic voting was found to be inaccessible according to eCH-0059. Therefore, we collaborated again to identify solutions. This process occurred over several iterations until we were able to write a report and publish a formal letter of accessibility compliance according to eCH-0059 at the AA level.

4.1.5 Evaluation

We point out that the action and evaluation stages are iterative and joint. During the evaluation phase, the action research team collaborated to evaluate the results of tasks performed (action). We conducted a two-step process: external audit and challenges to the answers to our research question. Throughout our process, we wrote a memo recording the status of our emergent thoughts.

4.1.6 Learning

Despite our certification, the final accreditation was subject to the Federal Chancellery judgment. In February 2018, one month prior to the deadline, the Federal Chancellery certified the electronic voting IS.

Even with this success, we realized that the action-oriented research was incomplete. Indeed, the standards made it possible to evaluate accessibility for PVI. However, the standards pertained primarily to the technical aspects, unfortunately ignoring the service-related needs of all voters [69]. Humans are the focus of services, and end users must be able to interpret the information and to perform the intended processes and activities using digital resources [70]. In this respect, we confirm the results of previous research that suggests performing an evaluation beyond accessibility guidelines [34, 37, 46].

Inspired by the communication theory of Shannon and Weaver [71], we hypothesize that accessibility is only the first technical level necessary to transmit and interpret information (Fig. 2). For these pioneers of communication theory, communication problems can be identified on three levels. The technical level considers the accuracy of information transfer between the transmitter and the receiver. The semantic level is focused on the interpretation of meaning by the receiver. The influence level is concerned with the effectiveness of the message in influencing the receiver's behavior. We point out that the influence level is the result of the process and seems, in our opinion, to have little to do with the notion of the digital divide and to be less likely to help us answer our research question.

Fig. 2
figure 2

Levels of communication problems [71]

Full size image

In our context, if accessibility does meet the level AA, we need to address the semantic level of information. Thus, we question to what extent the digital information of the electronic voting IS is understood by PVI. We identified this research question as the entry point for another cycle of our action research.

From a technical point of view, we found that it was complicated to modify an IS that is originally inaccessible according to standards. We can consider that this is complicated for reasons of development time and effects related to the software entropy principle. Therefore, we called upon practitioners to integrate accessibility principles (i.e., accessibility by design in a proactive approach) right from the IS design phase.

4.2 Second cycle: semantic intelligibility of the information of the electronic voting IS for PVI

4.2.1 Research environment

In collaboration with our partner, we agreed on a specific aim for the second AR cycle: evaluate to what extent the digital information of the electronic voting IS is understood by PVI (semantic level). This aim enables us to feed our research question. In the second research, we integrated two blind end users into the research team to assess the semantic intelligibility of information of electronic voting IS in a more holistic way, beyond the standards and IT scope.

4.2.2 Diagnosis

The diagnosis of this second cycle was based on the evaluation and knowledge acquired during the first cycle. Indeed, standards-based accessibility of electronic voting IS was deemed insufficient for PVI.

4.2.3 Action plan

To assess the electronic voting IS beyond technical standards, we built an instrument to evaluate the semantic intelligibility of information (evaluation guide). Therefore, we planned new activities for the implementation and evaluation phases:

  1. (1)

    Build a guide to evaluate the intelligibility of information of this IS.

  2. (2)

    Evaluate the intelligibility of information (by PVI).

  3. (3)

    Conduct interviews with PVI.

  4. (4)

    Process qualitative data collected.

  5. (5)

    Propose an analysis of the semantic intelligibility of information of the electronic voting IS.

4.2.4 Action

We qualitatively evaluated the intelligibility of information of the electronic voting IS, because it was not possible to question a sufficient number of PVI. To process the qualitative data collected, we relied on NVivo which tends to increase the rigor of the methodological process [72]. Our use of NVivo was driven by the operational coding approach proposed by Corbin and Strauss [73]. Beyond the coding, throughout the data analysis, we wrote a memo that traces our reflections in order to feed our answers to our research questions.

4.2.5 Evaluation

During this second cycle, we invited the two PVI respondents to evaluate the full electronic voting process and thus simulate a vote on the electronic voting test platform. We made sure to offer the PVI the same conditions as on the production platform. Thus, we provided the two PVI a paper mail containing a code to access the electronic voting test platform. We conducted the evaluation by instructing the two PVI to use the open-ended questions from our instrument (the guide is available upon request to the authors). In addition, we asked them to write down their comments on the intelligibility of the e-voting system. We collected these comments through telephone interviews. We thoroughly analyzed their responses, allowing us to develop a complete observation of the intelligibility of information of the electronic voting IS.

4.2.6 Learning

The required information that was to be presented to the voter was indeed accessible. However, at a semantic level, it was not always intelligible, whereas intelligibility is a precondition to comprehension [74]. The IS did not meet the need of autonomy of people with disabilities and negatively impacted usability.

Here, we present two specific examples. First, a PVI citizen, like everyone else, receives a letter (paper mail) containing a code (in text) to be used for electronic voting. Unfortunately, it was very difficult for PVI to locate this code in the letter, disenfranchising them as a group. Although the IS was technically accessible, the overall process was not. Secondly, the PDF document presenting the cantonal objects submitted to a vote is not completely intelligible. Some diagonally formatted texts cannot be interpreted using assistive technologies.

4.3 Third cycle: technical accessibility and semantic intelligibility of the electronic voting IS

The third cycle is a confirmatory one. Thus, the third research environment was identical to the second and the process is the same as those of the first and second cycles. In this context, we evaluated the technical accessibility and intelligibility of a PDF document presenting the cantonal objects submitted to a vote. In collaboration with the canton, we have improved the technical accessibility and, in a second step, the intelligibility of the PDF document. We then interviewed two PVI to assess the accessibility and semantic intelligibility of the document. Thanks to technical and then semantic improvements, the IS better met the autonomy needs of PVI, without affecting its usability.

4.4 Discussion of the main findings

The cycles of AR conducted generated findings and allowed us to acquire new practical and theoretical learnings. In accordance with the precepts of AR, we synthesize the practical learnings and findings in Table 2 and the theoretical findings in Table 3 as actions of our research in Subsects. 4.1.6 and 4.2.6. We also deepen our analysis of the main findings before discussing them. The discussion is focused on theory for design and action that say “how to do something” [16]. In that sense, we attempt to discuss which accessibility evaluation methods are suitable and how to include them into the development process of an accessible e-government IS.

Table 2 Synthesis of the practical learnings and findings
Full size table
Table 3 Synthesis of the theoretical learnings and findings
Full size table

Performing an RCCA aims to shed light on theoretical pillars of a research field [11]. Regarding web accessibility in e-government, we found that the standard approach refers to an automatic accessibility evaluation. While the e-government literature frequently discusses accessibility and usability [49, 50], little research of our set promotes a wider perspective than accessibility only [52, 53]. For instance, researchers customized their evaluation practices considering multiple methods and stakeholders [53] or evaluated other dimensions (e.g., usability and security) [52]. When the focus is placed on PVI, there is a lot of criticism about such norms [34, 37, 46], particularly about WCAG 1.0 and 2.0 which has been the most used. This finding highlights the importance of involving people with disabilities, without which real accessibility problems cannot be identified [41, 45, 53]. We recall that at the end of our first AR cycle, we felt a sense of incompleteness of only conducting a checklist-based accessibility evaluation to obtain a federal certification. As stated by Fagan and Fagan [49] "accessibility it’s more than just the law". These findings open the debate of which perspective is suitable in our context. We could say that an accessibility perspective is not sufficient. Several works favored a usability perspective when PVI are involved [37,38,39, 42, 45], or at least claims for a user-centered accessibility one based on usability methods [41, 49]. In this line of research, one can find the Quality in Use Integrated Measurement (QUIM) model [75] that consolidates 10 usability factors, while one of them is accessibility. Moreover, Sauer et al. [76] attempted to unify accessibility, usability and user experience toward an integrative model and a higher-level concept termed as interaction experience. This approach recommends performing the three kinds of evaluations and aggregates them to obtain an entire view of a system in use. More recently, in the WCAG 3.0, actually in draft, we noticed the apparition of holistic tests [77]. Holistic tests include assistive technology testing, user-centered design methods and both user and expert usability testing. Successful results of holistic tests are used to reach a silver or gold rating.

Regarding the process of developing accessible e-government services, the core of web accessibility used by scholars can be generally found in normative documents [29, 30] in whatever the subfield we investigated. However, prescriptive documents say what to do and not how to do the work. In an e-government context, some practical guidelines can be found in the recommendations or implications sections of several works (cf. Table 1). We should specify that such contributions, which aim to improve web accessibility awareness, practices and policies, are often the last contribution of several ones in a paper. This implies that such guidelines are, in our opinion, insufficient to reach a gold standard. Some details about theory for design and action [16] can be found in three works of our set. We can highlight the Web Accessibility Integration Model developed by Lazar et al. [35], highlighted by two RCCAs, the 32 guidelines related to screen reader users of Theofanos and Redish [37], and the accessibility fixes prioritization developed by Al Mourad and Kamoun [48]. While Lazar et al. [35] briefly present a design of an initial website which is re-designed in a further step, further details about the process can be found outside the works highlighted by our RCCAs. For instance, Billi et al. [78] developed and operationalized a unified methodology for the evaluation of accessibility and usability of mobile applications. In their approach, accessibility precedes usability. Expert evaluators that can be PVI are involved in the second phase, for instance in a heuristic evaluation.

The open discussion about accessibility methods, processes and practices motivates the need to provide a conceptualization and prescription of how to do the work. Moreover, a method could take into account the evolution of the literature [78] and policies [55]. Our learnings show that in order to bridge the digital divide for PVI, it is necessary to improve (1) the technical accessibility of information, and (2) the semantic intelligibility of information. Inspired by Shannon and Weaver’s communication theory, we argue that it is necessary to adapt the format of the information source to the recipient. This shortcoming implies that while the electronic state voting system is technically accessible, the full process is not. IS designers and developers have to consider the full business processes, from the information source to the destination, and not only focus on the technical aspect of websites. To emphasize this point, we recall that the paper mail containing the code to vote (in full text) does not allow PVI citizens to be autonomous. We also acknowledge that IS designers and developers must rely on ex ante standards during the design and development of websites. The active involvement of users (including PVIs) in the design and evaluation process is a prerequisite for a successful minority-inclusive project. Moreover, modifying an application ex post is difficult and could not reach accessibility, even at a minimal level.

5 Conclusion

Let us recall the words of Tim Berners Lee, the inventor of the Web, ‘The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect’. Such words are in line with the answer to our research questions. Compliance with technical standards cannot be the end goal; designers and coders must widen their technical visions by verifying the semantical intelligibility of information in IS.

Our study provides empirical evidence that technical standards are not sufficient to support PVI to overcome the digital divide. The use of technical standards guarantees accessibility at a technical level. Considering the feedback of PVI on semantic intelligibility of information helps to improve IS and to overcome the digital divide.

One of the limits of this research is that we relied on a single case. Moreover, in the context of an electronic state voting system, information is of key importance for democracy. The semantic intelligibility of information is of paramount importance. Hence, it would be appropriate to replicate this study with an IS containing fewer sensitive data. In such a case, technical standards might be sufficient to support PVI to overcome the digital divide. Finally, future research could extend our approach to identify and distinguishing technical and semantical causes of digital divide.