Keywords

1 Introduction

The Internet has become commonplace for a significant proportion of the world’s population (40 %) [1], for communication, information searches or the purchase of goods, among many other uses. This mobile period ensures that more and more people have access to the Internet on mobile devices. According to Global Web Index [2] 80 % of adult Internet users in the world have a smartphone and almost 50 % a tablet. The increasing development of wearables supports this trend.

Also more and more senior people have discovered the World Wide Web (WWW) as an information and communication medium with many new possibilities. Unfortunately, not everyone has been able to benefit. People with physical or mental disabilities often come up against barriers that make it more difficult or impossible to use the WWW. One example being insufficient contrast between font color and background color or too small font sizes. There is, therefore, a necessity to make web pages more accessible. Also people without physical or mental disabilities benefit from better web page accessibility.

In the international standard Web Content Accessibility Guidelines (WCAG) 2.0 the guidelines for the implementation of accessible web pages are defined and regulated [3]. Single automated test tools partly exist to test these guidelines, such as for checking color contrast, alternative texts or similar. However, for some automated testing is not technically possible, such as to check whether the header structure has been set up correctly and logically or meaningful alternative texts were used for graphics (success criterion 1.3).

The aim of this study is to develop a web-based test-procedure as a prototype, which the WCAG 2.0 guidelines supports. As a result, the tester obtains a statement whether, and to what degree, the guidelines of accessibility are fulfilled on the web page. In the first step, the test procedure is limited only on web pages that are accessible on a desktop PC. In the future, it should be possible to check the accessibility for invoke web pages on mobile devices (smartphones and tablets).

2 Legal Agreements and Standards

“The World Wide Web Consortium (W3C) is an international community where Member organizations, a full-time staff, and the public work together to develop Web standards” [19]. The W3C has published more than 80 standards [6] respectively recommendations, such as HTML, CSS, XML and also the WCAG. This recommendations explain in detail how web pages can accessibly be developed, so that people with disabilities can use the web pages without barriers. It was implemented at the national level in many countries around the world. The Web Accessibility Initiative (WAI) is a working group of the W3C, which deals with the accessibility of web pages, and was significantly involved in the development of WCAG.

Before the structure of the WCAG is described in detail, it is first necessary to define accessibility. The WAI define accessibility as follows: “Web accessibility means that people with disabilities can use the Web. More specifically, Web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the Web, and that they can contribute to the Web. Web accessibility also benefits others, including older people with changing abilities due to aging” [12].

The authors of this article provide a more comprehensive definition of accessibility. The definition of the WAI includes only a certain group of people. The goal of accessibility it is not to offer web pages accessible only for people with disabilities, but also to allow any person to use a web page without barriers. There should be, therefore, an expanded definition of accessibility based on our paper:

“A web page is accessible if the following applies:

  1. 1.

    There is one access for all people to the content and functionality of this web page.

  2. 2.

    This access is equally well available to all people” [9].

This definition brings out the aim of accessibility clearly, because accessibility does not require special solutions for disabled people.

The latest version of WCAG was published in 2008 in the version 2.0 [3]. It was comprehensively revised compared to WCAG 1.0. The differences can be found under [7]. However, the guidelines and success criteria were written in a technology-neutral form and do not contain instructions or programming examples. These are found only in the non-normative documents “Techniques for WCAG 2.0” [5] and “Understanding WCAG 2.0” [4]. So the WCAG is open to new future technologies [8].

The WCAG consists of four principles (perceivable, operable, understandable, robust). These are further subdivided into 12 guidelines that describe the goal accurately. Each guideline contains testable success criteria, which are each associated with a levels of conformance: A (lowest), AA, and AAA (highest). Altogether there are 61 success criteria. In Fig. 1 an overview of the structure of the WCAG 2.0 is shown.

Fig. 1.
figure 1

Overview of the structure of WCAG 2.0 (Color figure online)

Full size image

3 Test Procedure vs. Test Tools and Automatically vs. Manually Testing

3.1 Test Procedure vs. Test Tools

First we are going to explain, in general, the difference between the terms test procedure and test tools.

Test Procedure. In a test procedure to verify the accessibility of web pages in individual test steps about all success criteria of WCAG 2.0. The test of individual success criteria is carried out either automatically or manually. As a result, it is possible to make a statement about how well accessibility by WCAG 2.0 has been met (degree of accessibility or a statement of levels of conformance). An example of such a test procedure is the “BITV-Test” [14]. Although this procedure is testing the German accessibility guidelines (BITV), these are based on the international WCAG 2.0.

Test Tool. In a test tool, one or more success criteria will test automatically or semi-automatically. Here there are different classifications in terms of platform [15]:

  1. 1.

    Online Service: “They work by having the site visitor input the URL of their web page, selecting from any evaluation options, and then selecting a “Go” button or some other method of initializing the program” [15].

  2. 2.

    Within a browser: Browser extensions, they provide extra menu options within the browser itself (e.g. as a toolbar).

  3. 3.

    Within an authoring tool: These tools have been created to function as part of a web authoring tool, such as Macromedia Dreamweaver or in content management systems. The user can examine their content in the same environment they are using to create this content.

  4. 4.

    Install on hard drive: The tools required can be installed on your hard drive. They are able to examine web pages locally.

Examples are extensive tools such as the European project “EIII- European Internet Inclusion Initiative” [13] or WAVE [16], as well as smaller tools such as ContrastFinder [22].

3.2 Automatically vs. Manually Test

For checking the accessibility according to WCAG 2.0, there are already many automated test tools. However, for the web-based test procedure presented here it is not intended that the test of the success criteria is exclusively automated. The reason for this is; when testing accessibility the human inspection plays a major role because not all success criteria are automatically tested. The following example demonstrates this.

<img src   =   “image/university_logo.jpg’’alt   =   “Logo of the university of Magdeburg’’/>

The alt attribute (alternate text) is described the pasted image. This is especially important for people who use a screen reader usually blind people. The screen reader reads the content of the alt attribute, so the user knows what is to be seen in the picture. Although an automatic test tool can verify the existence of an alt attribute within an img element, it cannot verify whether the description in the alt attribute fits to the displayed image (success criterion 1.1.1). This can currently only be checked manually [11].

For this reason, the checking of a web page for accessibility must be made in two stages. Firstly, automatic or semi-automatic test tools are used, further an (extra) manual check is carried out on other success criteria by the tester.

4 Empirical Study of Existing Test Tools

4.1 Classification of Automatic Test Tools

For the developed web-based test procedure existing automated test tools for checking individual success criteria of WCAG 2.0 shall be used. An empirical study has been used to evaluate which tools are appropriate. First, a classification of test tools was carried out. Figure 2 shows a subdivision of automatic test tools in three classes [12].

Fig. 2.
figure 2

Overview of various types of automated tools

Full size image

  • Adjustment tools: With the help of adjustment tools automatic corrections can be carried out. If an automatic correction is not possible, these tools help the user step by step to carry out any necessary corrections. An example of such an adjustment tool is Tidy. Tidy validated firstly the (X)HTML source code and then shows necessary corrections.

  • Filter- and transformation tools: With these tools, the page viewed is changed or added directly in the browser. For example, color simulations can be carried out which make it possible to look at a page as they would appear to a person with a certain color vision anomaly. Often these tools are integrated as a browser plug-in .

  • Test tools:

    • General test-tools: The aim of these programs is to find many different barriers. However, the barriers found are often the same, since only a part of the guidelines are automatically tested. One example of this would be the EIII project [13]. With the help of a tabular overview the success criteria that were not met have been explained.

    • Specialized test-tools: For these programs only one or a small number of barriers is checked. Frequently, these test tools are better implemented, as is the case with the general test tools, because they are usually very complex. For example, the test tool “Colorblind Web Page Filter” allows to see how an existing web pages appear to colorblind users [26].

    • Service tools: So far there are very few services. An example of such a service would be the AccMonitor (no longer available) of a given website. This tool constantly monitors and alerts the user with updates regarding barriers, for example via email.

4.2 Evaluation Criteria for the Evaluation of Test Tools

In order to evaluate which of the automated tools is suitable for testing of the success criteria according to WCAG 2.0, the following evaluation criteria were established.

  • For WCAG 2.0 relevant?: The test tool must check at least one success criterion in WCAG 2.0. There are many old tools that examine, for example, only the guidelines according to WCAG 1.0 and would therefore not be suitable.

  • Freely available?: Checking accessibility with a tool should be possible for each user. Financial barriers for automatic test tools should be avoided for this reason. Therefore, all commercial tools are ruled out. Only freely available and open source tools should be used.

  • Suitable for web pages?: Only web pages on personal computers will be monitored and evaluated (see Introduction). Only tools for this purpose come into consideration. Tools that only check other techniques cannot be considered, such as accessible PDF documents.

  • Which platform (online/plugin/Web Editor/HDD): Sect. 3.1 (Test procedure vs. test tools) explains the different classifications regarding the platform. Additionally it was determined that under the classification “disk” only tools which can be installed on Windows 7 or higher shall be used. In the classification “plugin” only tools which are run in the browsers Mozilla Firefox, Google Chrome or Microsoft Internet Explorer come into consideration.

  • Easy installation: Here the criteria is whether the installation and also the

  • general handling of the tool is possible without problems. Possibly, individual decisions must be made.

  • Language: The tool must support, at least, the English or German language. This is because the test procedure will initially be used in German-speaking countries.

If all quoted evaluation criteria are met, the evaluated test tools are approved for the developed test procedure.

4.3 List of Evaluated Test Tools

In the course of an empirical study the test tools were specified and checked according to the evaluation criteria in Sect. 4.2 (Evaluation criteria for the evaluation of test tools). A basis of the examined test tools is provided in the list of the W3C [10] with a total of 38 tools. The licensing of the tools extends to “Commercial”, “Enterprise”, “Free Software”, “Open Source” and “Trail or demo”. Because of the determined evaluation criteria only “free software” or “open source” tools are considered for this investigation. Generally, these tools can automatically test one or multiple success criteria of WCAG 2.0. Also, additional tools that were eligible for evaluation were researched through internet search engines. A total of approximately 100 test tools were evaluated. The presentation of the complete list of all examined testing tools cannot be shown here for reasons of space. Therefore in Table 1, only an excerpt of the results is presented.

Table 1. List of evaluated test tools
Full size table

5 Specification and Functionality of the Test Procedure

5.1 Specification and Validation Model

Specification. First, the functional, qualitative, platform-related and process-related requirements were specified in the test procedure. The test procedure differentiates between the two user groups “authenticated” and “non-authenticated user”. An authenticated user had significantly more functionalities as compared to a non-authenticated. It can create and select new projects, evaluate and store success criteria, view an evaluation and change and store their personal information. The term “project” refers to the examined web page being checked using the test procedure in compliance with the success criteria. Figure 3 gives a summary overview of the functionalities for an authenticated user. Initially a non-authenticated user must register to use the test procedure.

Fig. 3.
figure 3

Use case diagram for an authenticated user

Full size image

Validation Model. The fulfillment of success criteria can either be measured by a test tool or (if no tool is available /possible) assessment by the tester. Finally the tester indicates for each of the 61 success criterion if it is “fulfilled”, “not fulfilled” or “not applicable”. The quality and reliability of the obtained results depends to a large extent on the test tool and the assessment by the tester. Now, this validation provides the basis for the establishment of a numerical value, the degree of accessibility. Of course, the calculation of such a value will certainly lead to discussion, because the validity of such a value does not tell much about how accessible a web page really is. However, this value should only be used as an additional indicator to assess whether the efforts to implement an accessible web page are on the right track.

A weighting of the success criteria (Level A is more important than Level AAA) will not be considered for the model, because all success criteria are equally important. Therefore, the degree of accessibility is calculated as follows:

$$ {\text{Degree of accessibility }} = \, \frac{{\left( {{\text{count fsc }} + {\text{ count nasc}}} \right)*{ 1}00}}{{ 6 1 {\text{ sc}}}} \, $$
(1)

sc = success criteria

fsc = fulfilled success criteria

nasc = not applicable success criteria

The result calculated will be a value between 1 and 100. A 100 percent accessibility is not possible, therefore, the following classification (Fig. 4) shows the tester how accessible the tested web page is. This classification is based on the BITV test procedure [17].

Fig. 4.
figure 4

Classification of the evaluation of the valuation model

Full size image

5.2 Functionality and Operation

The workflow, as an authenticated user goes through the procedure for testing a web page to WCAG 2.0, is provided as follows. After authentication by logging in with a username and password, the user creates a new “project” or selects an already created project. Now the user will see all success criteria in the main menu that have to be tested in detail sequentially. For each success criterion the following information is provided:

  • Meaning of success criteria: If the user does not understand the meaning of the success criterion, it is, at this point described in detail in order to avoid incorrect implementation of the success criterion.

  • Information about more references/sources: If the user wants information about more references and sources, it will be offered here. The web pages of W3C are the most frequently used sources.

  • Test tool and procedure for evaluation: Information about the test tool that is used for the evaluation and the procedure of the evaluation itself.

  • Examples: Examples describing the possible situations a user may encounter help in the technical implementation.

  • Evaluate: If all previously collected information for the tester is clear, he/she can start to evaluate the success criteria using the specified test tools. For this, depending on the success criterion, a number of different questions are available. These questions detect and evaluate all aspects of this criterion. The answers to each question are selected with a drop-down menu (fulfilled, not fulfilled or not applicable) and stored using the “Save” button in the database.

6 Implementation of the Test Procedure

The developed test procedure was implemented as a web-based prototype. A part of the success criteria was selected, to illustrate the test procedure by way of example. The structure of the web page is presented as an overview in Fig. 5.

Fig. 5.
figure 5

Design of web-based user interface of the test procedure (Color figure online)

Full size image

The three areas header, navigation and content were further specified (1). The navigation is divided into three navigation areas. The first navigation area (2) includes topics on which any user has access, independent of his user group. For example, to get information about the workflow of the test procedure. On the second navigation area (3) there are web forms to authenticate a user. The user can login or register for example. If the authentication is not possible due to incorrect data entry, you will get an error message at this point. If the authentication is successful, then the third navigation area (4) is displayed and for unauthorized users this area is hidden. In this area all the success criteria are listed. Next to each success criterion a traffic light icon in the navigation is visible that is intended to illustrate to the user the status of the verification at a glance (green = fulfilled, red = not fulfilled, yellow = Not applicable, cross = not checked).

The content area can reveal different forms (5), for example, a registration form, contact form or a form to edit the user data. The main function of the test procedure is the provision of the questions (6). Each question is answered by means of a drop-down menu. Included are the above mentioned possible answers. Additionally, information is provided for evaluation (see Sect. 5.2 “Functionality and operation”). After pressing the save button, a notice appears about the successful save in the database or an error message. Furthermore, an evaluation of the results (7) is calculated and generated dynamically. This provides information on the achieved level of accessibility.

7 Summary and Outlook

It was developed as a prototype web-based test procedure, which initially checked parts of the success criteria of WCAG 2.0. The human tester is supported by the use of automatic or semi-automatic testing tools. As a result, the degree of accessibility is determined, which gives information about how well the success criteria on the tested website has been respected.

To prepare the prototype for production, further work is needed. First, a full implementation of all the success criteria must be carried out in the test procedure. Moreover, it is desirable that national legislation (for example, the American Secion508 [13] or the German BITV 2.0 [14]) be checked with regard to this test procedure. The national guidelines are similar to the international WCAG to a very large degree.

After complete implementation extensive tests must be performed to check the validity.

In a later phase, the test procedure could be adapted to Accessibility Conformance Evaluation Methodology (WCAG-EM) [18]. In this non-normative document recommendations are given to evaluate a test procedure’s conformance requirements of WCAG 2.0.