1 Introduction

Today the Internet is an essential way to access information and services. People use it for communication and getting information. The continuous online presence has resulted in the availability of more and more government services on the web following the private sector. So that people with disabilities can have easy access to these services, it is important that websites are easy to use, accessible, and properly secure. To fulfil these conditions, we have set out three main requirements that are strongly recommended to every website owner.

Usability is one of the most important requirements of websites, which also plays an important role in public websites. This indicator shows how easily and efficiently the website can be used to access the desired service. If a website is too slow or too complicated to manage, it increases user dissatisfaction and the risk of clicking away.

Another very important requirement is the accessibility of websites of public sector bodies, which allows persons with disabilities to perceive, understand, and interact with websites and their services. The European Union directive on accessibility of websites and mobile applications [1] obliges public sector bodies to comply with the requirements of WCAG 2.1 (Level AA) [2]. Compliance with the requirements helps a wide range of people with disabilities, including blindness and low vision, deafness and hearing loss, learning disabilities, cognitive limitations, limited movement, speech disabilities, photosensitivity and combinations of these to gain access to websites and mobile applications of public sector bodies.

The third requirement is security, which is also essential, since users need to provide sensitive information to access each service. In the age of web content management systems, it is of utmost importance that communication is made through an encrypted channel, the system used on the website is up-to-date, and contains up-to-date extensions from a reliable source. If these basic security steps are missed, it will greatly increase the possibility of SQL injections, Cross-site scripting (XSS) and other website attacks, which may result in significant data loss.

The legislation passed in the European Parliament applies to all member states, including Hungary. According to the adopted directive, all websites and mobile applications of public sector bodies must be accessible by the end of 2021, otherwise the country may be penalized. In order to implement this process, a comprehensive study of the state of Hungarian government websites is required as a first step. Although some studies [3,4,5] on Hungarian websites were conducted in the early 2010s, the results of these studies are outdated and some of them did not focus specifically on websites of public sector bodies. The aim of our study was to carry out a survey providing up-to-date information on the Hungarian government websites and to develop suggestions for the problems that may arise.

The remainder of this paper is structured as follows: Sect. 2 provides a review of the relevant literature. Section 3 describes the methodology used in the study. Section 4 presents the results of the study. Section 5 makes suggestions to improve the issues presented in Sect. 4. Finally, Sect. 6 presents the conclusions.

2 Literature review

2.1 European Union legislation

The providers of information and services, such as public sector bodies, rely increasingly on the internet in order to produce, collect, and provide a wide range of information and services online which are essential to the public. For this reason, it has become important to develop a directive that describes accessibility principles and techniques, that should be followed when designing, building, maintaining, and updating websites and mobile applications, in order to make them more accessible for users, in particular for persons with disabilities. On 22 December 2016, the European Union directive (2016/2102) [1] on the accessibility of websites and mobile applications of public sector bodies came into force, that provided persons with disabilities with better access to the websites and mobile applications of public sector bodies. Until further provision, the directive is based on the requirements of the harmonized European standard on Accessibility requirements for ICT products and services (EN 301 549) [6]. Chapter 9 of the standard requires websites of Information Communications Technology (ICT) to fulfil all five requirements of WCAG 2.0 AA level [7]. Starting with version 2.1.2, the standard requires WCAG 2.1 AA level [8]. WCAG 2.1 includes all the eligibility criteria for 2.0 with 17 additional criteria [9] on mobile accessibility, people with low vision, people with cognitive and learning disabilities. As a result of this directive, all new websites of public sector bodies of EU Member states released after 23 September 2019 should be accessible, and after 23 September 2020, all previously created websites of public sector bodies should be accessible as well. After September 23, 2021, all mobile applications of public sector bodies must be accessible.

2.2 Usability test for public sector bodies websites

During our literature review we found various types of studies on websites of public sector bodies. One of the main types of the reviewed papers examined only the usability of the pages. Usability refers to how effective users are in using the site and how easily they access the content. Usability analysis basically uses two types of testing approaches: analysing all pages of a website or focusing only on the home page of a website. Home pages are often the most accessible pages of a site and developers usually pay attention to this. Numerous studies [10,11,12] have shown that there is a correlation between problems detected on a home page and other pages of the same website. This is especially true for content management systems, where the structure of the website is generally fixed and only the content is different. Most articles used automatic online tools. Stowers [13] defined six criteria to explore usability issues. These are the following: online services, user assistance, navigation, legitimacy, information architecture, accessibility. In [14], it has been shown what are the main factors that affect the usability of e-government websites. These include navigation, efficient readability, the use of images and graphics, multimedia technology, page structure, and the ability to search for information. Huang and Benyoucef [15] have defined 13 guidelines and 13 criteria to measure the usability and credibility levels of e-government sites. Choudrie and Weerakkody [16] have tested numerous e-government websites of several countries (Singapore, Finland, Canada, Hong Kong, Australia) with the WebXact tool. The analysis included testing faulty links, download times, elapsed time since the last update, style sheets, server-side image maps, internal multimedia elements, and metadata elements, browser compatibility issues, and HTML verification errors. The results showed that many governments ignore accessibility and quality criteria. Islam et al. [17] examined the usability of 6 e-government websites in Bangladesh with the help of 22 graduate students. The results showed that the tested websites had many critical usability issues, which need to be improved as soon as possible.

2.3 Accessibility test for public sector bodies websites

Accessibility and usability are closely related aspects in creating websites that work for everyone. Their goals, approaches, and guidelines overlap significantly. Web accessibility means that people with disabilities can equally perceive, understand, navigate, and interact with websites. The EU directive [1] also applies to this. Like usability analysis, accessibility analysis uses two types of testing approaches: analysing all pages of a website or focusing only on the home page of a website. There are several methods to check the accessibility of a website, one of these is the use of online checkers. Abanumy et al. [18] studied the e-government pages of Saudi Arabia and Oman with various online tools. Among other, the Multiweb, the LYNX, and the W3C checkers were used to examine how websites interact with different input devices, the support of text-based browsing, and the presence of HTML syntax problems. Shi [19] discovered a number of major accessibility issues in the Chinese e-government websites, and some minor bugs in the Australian government websites. A year later [20], Shi analysed 324 local Chinese local government websites, many of which repeatedly violated the W3C accessibility guidelines. Many similar surveys [21] were conducted in Europe (England, France, Germany, Switzerland), Asia (China, India, Cambodia, Philippines) and finally in Africa (South Africa, Libya, Namibia, Kenya). The results show that most of the tested websites did not comply with WCAG 1.0 verification standards. Kopackova et al. [22] have analysed 39 websites of Czech public sector bodies based on civic aspects and search engine discovery. They found that the government websites contain deficiencies, which make it difficult for users to find and display the information they need. Al-Khalifa [23] examined 36 Saudi Arabian websites of public sector bodies based on accessibility requirements. As a result of the research, none of the websites were found able to complete the WCAG 2.0 conformance test in 2010. Research has shown that a high percentage of website errors are level A errors, and serious efforts should be made to ensure that websites fulfil the minimum requirements. Al-Khalifa et al. [24] in 2017 re-examined the Saudi Arabian websites of public sector bodies. Analysis has shown that over the past 5 years, the number of violations of WCAG 2.0 rules has been drastically reduced, and more and more websites have supported mobile appearance.

2.4 Hungarian unique practice, the segregated web accessibility

Unfortunately, segregation due to disability still exists today. When people with disabilities are asked to use a separate interface to access the information on a given website, it is basically one type of discrimination. Many studies have already been conducted on the negative effects of segregated accessibility. Lazar et al. [25] have explained that segregated solutions typically navigate disabled people with limited functionality and less up-to-date information. A typical example of this practice is shown in Figs. 5 and 6 in the “Appendix”. According to Lazar, modern programming techniques allow creators to design websites that are accessible to everyone without segregation. The designation of segregated web accessibility is unique in Hungary. In this practice, the accessible version of a website is indicated by three black dots on the yellow circle as shown in Fig. 1. This designation is currently being used in several countries to support people with disabilities (e.g. at road signs), but as far as we know, regarding websites, it is used only in Hungary. However, an “accessible version” of such websites usually only concentrates on the contrast only, it has black background and increased yellow letters. But on the other hand, these pages mostly have limited functionality, they are rarely updated, and do not fulfil the WCAG requirements. In our view, this bad practice may also have contributed to the lack of accessibility of Hungarian websites.

Fig. 1
figure 1

The icon for the accessible version of websites in Hungary

Full size image

2.5 Website security tests

In addition to usability and availability, the third aspect we consider important is security. Although security is a bit farther from the previous two aspects, the security of websites is at least as important as usability and accessibility. A Content Management System (CMS) [26] based website is basically considered secure if the communication is via an encrypted channel, the data is properly encrypted, and the parts of the website (base system, installed extensions) are up to date. Website security is an important topic, so many publications have been made on this. The Open Web Application Security Project (OWASP) has defined [27] the ten most serious security threats to web applications. Choudrie et al. [16] have investigated data protection using the WebXact tool on websites of public sector bodies in many countries. The results showed that no portal used encryption. In this study, it has been shown that 81.6% of 212 e-government websites contain some XSS and SQL injection vulnerabilities [28]. Shteiman [29] described the importance of upgrading CMS. Nowadays, content management systems have become very popular [30], therefore they have become popular targets for hackers. Many website owners do not keep up to date with their CMS and installed extensions, making them vulnerable to various hacker attacks due to public vulnerabilities. Abu-Shanab et al. [31] have reviewed Jordan's e-government websites using the Rapid7 monitoring tool. Studies have shown the websites under investigation contain a number of vulnerabilities that require a quick solution. Sahu et al. [32] have discussed the issue of safe web application development. It is recommended that more secure web applications should be created in compliance with the Secure Coding Standards.

2.6 Comprehensive website tests based on accessibility, availability, and security

The three aspects outlined above are important in themselves, but in a comprehensive study it is advantageous to consider them together. There are studies that analyse websites in specific countries based on accessibility, availability, and security. Ismailova [33] has analysed 60 Kyrgyz e-government websites using online diagnostic tools based on usability, accessibility, and security considerations. The survey showed that 46% of websites contained some usability issues and 69% had some accessibility issues. In the final phase of the study, it turned out that 55% of Kyrgyz websites used an outdated version of a content management system that contained public vulnerabilities. Noe [34] examined 79 Tanzanian e-government websites for usability, accessibility, and security aspects. The analysis used automated diagnostic tools such as Pingdom, Google Speed Insight, WAVE, W3C Checker, and Acunetix. The results showed that 100% of the websites had broken links in terms of usability, and in 52 out of 79 websites, it took more than 5 s to load. Accessibility results showed that none of the web sites were able to fulfil the WCAG 1.0 policies. The results of the security analysis revealed that 40 of the 79 websites tested had one or more high-severity vulnerabilities (SQL injection, Cross site scripting). Based on the results, the study makes some recommendations to improve the usability, availability and security of public institutions in Tanzania. Arvinder et al. [35] have analysed 280 hospital websites in many Indian cities for accessibility, accessibility, and security. Data were collected using automated tools (WebSiteOptimization, Measure Text Readability, TAW) for the survey. Accessibility analysis showed that 280 pages had a total of 22,491 errors. The minimum error was 3 and the maximum was 840. In the first part of the usability tests, the authors examined the readability of hospital websites. Websites were ranked on a scale of 0–100, where higher values meant better readability. Pages received an average of 46.7 of the Flesch-Kincaid readability score, which was the “Difficult to read” level. In the second phase of usability tests, many page parameters (page size, number of objects, total image size, page load time) were examined. The results showed that websites have an average size of more than 20 MB and contain an average of 117 objects, which significantly increased the page load speed. A security analysis revealed that 70 of the 280 websites examined were CMS-based. Some 25% of them had security issues due to non-upgraded software components.

3 Method

Previous studies [3,4,5] have shown that more than 90% of Hungarian websites are not accessible. Due to the limited implementation period of the EU Directive, new information was needed to determine the current national outlook. At the outset of the study, the assumption was that the websites of public sector bodies have not improved significantly from previous investigations.

The aim of the study is to test the most important Hungarian public sector websites. Hungary is divided into 19 counties. The websites of these county centres, 5 cities with county rights and the website of the Government of Hungary were investigated. Although the counties are subdivided into a total of 174 districts, their detailed examination goes beyond the scope of the current article. This study specifically focuses on the home page as a metric for web accessibility in general.

There are various web usability, accessibility, and security monitoring tools available on the Internet [36,37,38]. In the preparatory phase of the study, the selection of the appropriate tools was carried out based on various tests and evaluations. After that, with the help of the selected tools, we carried out a complete examination of 25 websites of Hungarian public sector bodies.

3.1 Evaluation of websites of Hungarian public sector bodies from usability aspects

Usability has various definitions. Some define usability in a literal sense, how logical the website is, how easy to find the information, how easy to navigate between the pages. The testing of this type of usability is difficult, because the definition is subjective, and the test can only be performed manually. We use another definition of usability, which focuses on measurable properties like heatmap [39], scrollmap [40], page load speed, etc. In this study, we focused on the page load speed analytics, because it correlates well with user retentions. Many tools were tested and according to our opinion GTmetrix is the most versatile one. GTmetrix [41] is a usability testing tool concerned with page speed analytics, which is based on various web performance optimization APIs (Google PageSpeed [42], Yahoo YSlow [43]), as well as parameters that affect usability (e.g. fully load time, total page size, number of HTTP requests). PageSpeed and YSlow are displayed on a scale from A to F, which is evaluated based on the classification of the corresponding rule of the optimization tool. After each test, GTmetrix displayed the score of the website and the percentage of each rule and compares them with the average values.

During the measurement, the test environment was set based on different parameters:

  • Test location The test computer (London, UK) closest to the Hungarian servers was selected;

  • Browser The most commonly used browser Google Chrome (62.0.3202.94), has been selected;

  • Connection Since broadband coverage in Hungary is adequate and the speed of mobile internet is at the forefront of the world [44], no restrictive parameters have been set for speed, so the unthrottled connection on the twisted pair was selected;

  • Versions of web performance optimization tool PageSpeed 1.15-gt1, YSlow 3.1.8.

The GTmetrix tool provides the following information:

  • PageSpeed Score Specifies the percentage of PageSpeed recommendations that a website can fulfil [45];

  • YSlow Score Specifies the percentage of YSlow recommendations that a website can fulfill [45];

  • Fully loaded time (s) Specifies the total number of seconds which is needed for the page content to load. Google recommends loading in up to 2–3 s in one study [46]. In our study, the average page load time of the websites checked by GTmetrix during the last 30 days was 7.2 s;

  • Total page size Page size also increases the page load time. In our study, the average page size of the websites that GTmetrix checked for the last 30 days was 3.1 MB;

  • Requests Specifies the number of HTTP requests for the page. The number of HTTP requests on a page is also directly proportional to the page load time. In our study, the average number of HTTP requests for websites checked with GTmetrix in the last 30 days was 88.

After the test environment was set up, all 25 websites of public sector bodies were diagnosed based on availability one by one. During the analysis, the results were manually entered into the central table grouped by error type [47].

Using GTmetrix is straightforward. In order to begin the investigation, the name of the website to be verified must be provided on the website of GTmetrix. After that GTmetrix displays the results in tabular form, as shown in Fig. 2. The Performance Scores window displays the cumulative evaluation of the API being tested, and the Page Details window displays a general summary (Fully Loaded Time, Total Page Size, Request). At the bottom of the page is a list of suggestions that are tested for each API.

Fig. 2
figure 2

Usability test with GTmetrix checker

Full size image

3.2 Evaluation of websites of Hungarian public sector bodies: accessibility aspects

There are a number of tools for checking the accessibility of a website. One of these, in fact the most flexible, is WAVE [48] online checker. WAVE is developed by the WebAIM [49], a non-profit organization. Due to the continuous development of the software, more and more WCAG 2.1 [2] rules can be verified. Another advantage of this tool is that it graphically displays those website elements which do not pass the accessibility standards. These problems are grouped into categories, and after clicking on the error, it can be seen which rule has been violated, and it gives a tip for solution.

Since 2019, WAVE has been able to validate websites up to WCAG 2.1 AAA levels, but we used an earlier version of WAVE in our study that had the following levels of compliance:

  • WCAG 2.0 A (basic level of the Web Accessibility Guidelines);

  • WCAG 2.0 AA (the second level of the Web Accessibility Guidelines, which includes level A compliance);

  • Full (WCAG 2.0 AA + recommendations of WAVE developers).

In the course of the study, we made accessibility checks on each of the 25 websites of public sector bodies. We used the Chrome plugin version of WAVE Evaluation Tool (1.0.9) with Full compliance test during the scan. At the time of analysis, the results were manually entered into the central table and grouped by error type [47].

WAVE is simple to use. Once installed, a green WAVE button will appear on the upper right side of the browser. This button must be pressed after loading a website. As shown in Fig. 3, WAVE summarizes the detected problems on the left side of the screen, while on the right shows these problems graphically. By pressing the <code> button at the bottom of the website, the source code of the website is displayed. After clicking on an error, WAVE details the problem and then jumps to the HTML source code to be repaired in the source code window. Based on this information, the repair of the page under review can begin.

Fig. 3
figure 3

Accessibility test with WAVE checker

Full size image

In the final phase of the analysis, for all websites of public sector bodies we investigated if the site uses an integrated or a segregated accessibility solution. We also checked whether the website contains an icon for the accessible version of the website, as has become a practice in Hungary (marked with three black dots on a yellow circle).

3.3 Evaluation of websites of Hungarian public sector bodies: security aspects

There are many web-based security monitoring software tools available on the Internet. A couple of such devices have been tested and the Sucuri [50] website inspector, a subsidiary of the American GoDaddy hosting company, was selected. In our view, this tool can retrieve the details of the websites in sufficient detail to help determine whether or not a particular website represents a security risk for users.

The Sucuri tool provides the following information:

  • Using TLS (SSL);

  • Web server type and version;

  • Website engine and version in use;

  • Programming system and version in use.

In the course of the survey, security checks were carried out one by one on all 25 websites of public sector bodies. During the analysis, the results were manually entered into the central table and grouped by problem type [47].

To start an experiment, the name of the page to be scanned must be entered. Figure 4 shows that after the check, Sucuri displays the results in different blocks of information. The top block contains the usual information on the website (IP address, Hosting, Server, CMS etc.). Figure 4 shows a graphical representation of the security rating of the website. Below further test results can be found (Website Malware & Security, Website Blacklist Status, Hardening Improvements).

Fig. 4
figure 4

Security test with Sucuri checker

Full size image

3.4 Evaluation of websites of Hungarian public sector bodies: other aspects

3.4.1 GDPR compliance

We used GDPR verifier [51] of the Danish Secure Privacy web security company to examine certain parameters of GDPR compliance [52] for websites of public sector bodies. The cookie compliance defined in the directive could only be checked manually because of its complexity.

The Secure Privacy tool provides the following information:

  • Prior consent on other than strictly necessary cookies (ePrivacy);

  • Prior consent on personal data (ePrivacy);

  • Personal data is transmitted to 'adequate countries' (GDPR);

  • Tracker identification.

3.4.2 Multilingual support

As a member of the European Union, it is important for Hungary to support the publication of content in several languages. As there were no tools available for automated evaluation of the languages available on the site under review, multilingual analysis was performed manually for all 25 public websites.

3.5 Limitation of online testing tools

The use of free automated testing tools is quite attractive as it is cost effective, easy to use, easy to access, and provides fast results. However, such tools have a number of limitations. Automated testing programs may mistakenly identify items as accessible or inaccessible, disregard the different disabilities or abilities of people with similar disabilities, fail to address usability or functionality issues, and omit accessibility issues that can only be identified by humans (keyboard access, meaningful alt texts). Automated software has an average error rate of at least 30%, depending on the test tool. Using automated tools can give the false impression that a good rating results in a completely accessible website [53]. While automated tools are not a complete solution, they are a necessary part of any serious accessibility evaluation effort.

For usability, we used the GTmetrix online checker. Pages submitted by users for analysing are processed in a queue, which may take some time. The tool focuses specifically on the page load time, so it cannot perform heatmap or scrollmap tests. A further limitation is that subscribers are required to use more detailed settings (device simulation, screen resolution, custom connection, etc.) when testing.

For accessibility, we used the WAVE online checker. WAVE's limitations include being able to test only one page at a time, not the entire website and the results are not exportable. The tool cannot check all of the issues in WCAG 2.1 guideline—no automated tool can.

For security, we used the Sucuri online checker. It may give a false sense of security when device indicates a low or minimal security risk because the version number of extensions installed cannot be verified when using a content management system. The device can only analyse one page at a time. Results cannot be exported.

4 Results

In this study, 25 websites of public sector bodies were examined, which include the government websites of the county seats and the most important districts of Hungary and the website of the Hungarian Government. A detailed list of the websites in the examination is given in of the “Appendix”. In addition to usability, accessibility, and security analysis, support for GDPR compliance and multilingualism has also been investigated. The results of each examination were systematized and then classified according to usability, accessibility, and security.

4.1 Usability analysis

4.1.1 Analysis of PageSpeed

In the first phase of the usability study, PageSpeed results were analysed. Table 1 shows how many websites have passed the PageSpeed analytics rules at different levels. An “A” rating means that the website has completed the PageSpeed recommendation fully or almost fully. The investigation revealed that only one website was able to achieve the highest rating, followed by three, and then the subsequent three sites achieved average results. More than half of the websites have reached the worst level of compliance.

Table 1 Results of PageSpeed analysis
Full size table

The 25 websites failed 20 different PageSpeed analytical rules fully or partly, with the 5 most common problems listed in Table 2. According to the survey, public sector websites do not have the images optimized and the browser of the visitor is not used to cache multimedia content. Nearly half of the websites do not optimize their images for mobile browsers, do not use GZip compression to speed the website load, and do not delay the loading of JavaScript.

Table 2 The five most common PageSpeed problems
Full size table

4.1.2 Analysis of YSlow

In the next phase of the study, we analysed the results of YSlow. Table 3 shows how many websites have passed the rules of YSlow analytics. Rating “A” means that the website has completely or almost completely fulfilled the YSlow recommendations. The investigation revealed that none of the websites could achieve the highest rating. Out of 25 websites examined, 19 were below average.

Table 3 Results of YSlow analysis
Full size table

The 25 websites failed 10 different YSlow analytical rules fully or partly, with the 5 most common problems listed in Table 4. Based on the survey, public websites of public sector bodies do not use content delivery networks (CDN) to store multimedia content. The majority of websites does not use CDN server for static contents, and cached content was not set to expire. Over half of the websites have over-frequent HTTP queries, and entity tags (ETags) configuration is missing without which cannot be ensured that the resource in browser cache and on the web server are identical.

Table 4 The five most common YSlow problems
Full size table

4.1.3 Analysis of fully loaded time

In the following analysis, the total load time of the websites was examined. The results are shown in Table 5. The study revealed that 9 websites achieved excellent results with a load time of 3 s, followed by 9 results achieving good ranking. For the last 7 websites, users have to wait more than 6 s to load the website, which significantly increases the risk of clicking away.

Table 5 Classification of website load time of websites
Full size table

4.1.4 Analysis of total page size

In the following analysis, we analysed the total size of the websites, the classification of which is shown in Table 6. The study revealed that 9 websites achieved excellent results with a load time of 3 s, followed by 9 results. For the last 7 websites, users need to wait more than 6 s to load the website, which increases the risk of clicking away.

Table 6 Classification of size of websites
Full size table

4.1.5 Analysis of number of HTTP requests

In the last section, we examine the number of HTTP requests on websites, which are listed in Table 7. Based on the results, it can be said that about half of the websites work with fewer than 80 HTTP requests. Out of the 25 websites examined, 7 have an average of 80 and 119 requests and 7 websites have to wait for more than 120 HTTP requests while browsing the government websites.

Table 7 Classification of HTTP requests of websites
Full size table

4.2 Accessibility analysis

4.2.1 WCAG compliance

In the first phase of the accessibility study, the WCAG 2.0 AA level compliance required by the directive was analysed. Table 8 shows that all of websites of the study contain at least one faulty item, so it can be said that none of them are fully accessible. We have found with the help of the WAVE results that more than half of the websites had relatively few errors, but 8 websites contained more than 30 faulty items.

Table 8 Errors in the WCAG 2.0 analysis
Full size table

The investigated websites ignored several WCAG 2.0 AA requirements. The five most common errors are shown in Table 9. Based on the survey, almost all websites contained empty links (does not describe the functionality and/or target of that link), more than half of the websites contained images that did not have alternative text, contained a form element without item description, or a button without description.

Table 9 The five most common WCAG 2.0 errors
Full size table

Table 10 shows the classification of the warnings received as a result of the WAVE verification. The results indicate that almost every website contains a warning. More than half of the websites had relatively few warnings, but 6 websites contained more than 30 faulty items.

Table 10 Warnings in the WCAG 2.0 analysis
Full size table

The five most common warnings received during the study are shown in Table 11. Based on the results, it turned out that almost all of the websites contained a link that is also reffered to by an adjacent element and a little less than half of the websites did not have an <h1> element and a heading level is skipped (e.g. an <h1> is directly followed by an <h3>.

Table 11 The five most common WCAG 2.0 warnings
Full size table

4.2.2 Compliance with recommendations of WAVE developers

In the next phase of the study, we have analysed accessibility compliance with the recommendations proposed by the WAVE developers. Table 12 shows that for more than half of the tested websites, the title attribute of a HTML link was identical to the text of the link. This unnecessarily slows down the browsing of the page by users who use screen reader software. In addition, 5 pages used justified format, which causes reading difficulties for people with dyslexia, and 2 pages used 10px or less font size that causes reading difficulties for people who have mild vision impairment.

Table 12 The three most common WAVE developer warnings
Full size table

4.2.3 Type of accessibility

In the last phase of the study, we have assessed the type of accessibility and the use of the icon navigating to the accessible version of the website. The study found that 16 out of the 25 tested websites were using Hungarian unique practice, the segregated web accessibility. As explained in more detail in the third paragraph of Sect. 3, it means that these websites have two versions, but unfortunately the accessible version most of the time is not updated and has less functionality. With the exception of one website, all segregated accessibility websites are indicated by three black dot-based icons on the yellow circle to navigate to the accessible version of the website.

4.3 Security analysis

During the security investigation, it was found that on the websites where the version of the web server, the engine of the website or the type of programming language were accessible, the versions that were in use contained an already known vulnerability.

In the first phase of the security investigation the distribution of web servers was measured. Table 13 shows that 23 out of the 25 public sector sites were able to query the web server type, 13 of which had the exact version number, all of which needed updating.

Table 13 Distribution of web servers
Full size table

Table 14 shows that the availability of the types of website engines was a little better, as only 8 of the 25 websites had their engines and versions publicly accessible. For websites where the version number is available, the engine contains an already known vulnerability that greatly increases the risk of hacking.

Table 14 Distribution of website engines
Full size table

For the programming languages, 11 of the 25 versions can be retrieved, each with a public vulnerability. The available version numbers all contained PHP programming language.

In the last phase of the study, the use of SSL websites was investigated. Our study revealed that only 10 of the 25 public websites used SSL encryption.

4.4 Analysis of other aspects

4.4.1 GDPR compliance

In this study, we used the Secure Privacy [51] tool to assess the compliance of GDPR directive [52] with the 25 public websites. The survey revealed that none of the tested websites fulfils the requirements completely.

The primary problem was that the websites did not provide a dedicated interface for configuring the cookies, where users can specify what type of cookies they allow while browsing the page, and the privacy policy did not include the categorization and detailed description of the cookies used on the pages.

It was also a common problem that none of the forms on the websites had a check box that would allow the site administrators to manage the data entered on the form.

4.4.2 Multilingual support

The pages have been tested for multilingualism, that 16 of the 25 websites contained at least one additional language. Table 15 also shows that more than half of the websites had an English version and in 9 cases also had a German version.

Table 15 Distribution of additional languages
Full size table

5 Suggestions

Based on the results we have developed several suggestions for improvement, which are presented in the following subsections. Since more than 57% [54] of websites on the Internet are CMS-based (and this number is constantly increasing), it is worthwhile to repeat the checks at least every 6 months, and when the main version of the CMS is changed, or when extensions are updated. Validation can be assisted by automated validation tools (which we have also introduced), but it is important to note that none of them can completely replace manual validation.

Based on our own experience, most of the problems can be traced back to the fact that governments are not fully aware of their legal obligations with regard to their websites (e.g. EU directives). Furthermore, they cannot provide sufficient resources to educate their content editors. The solution could be a national or EU partnership, where funding resources could be used to develop the necessary new methods and educate and train people. It is positive that many research and pilot projects [55,56,57,58] are currently underway with this objective.

5.1 Usability

When it comes to usability, PageSpeed and YSlow values of websites of public sector bodies, loading times, size, and the number of HTTP queries were carefully examined. Having completed PageSpeed checks, it turned out that more than half of the websites were below the acceptable level. Based on the five most frequent problems, we can conclude that using cache and Gzip page compression should be allowed in the content managing systems. In order to optimise image size, we recommend the use of images of a maximum necessary size. Various image compression procedures such as Karaken.io [59] and Optimizilla [60] are suggested. We also suggest the use of website optimising extensions for scaled images and to delay Javascript loading. One such solution is JCH Optimize, which is available on several top content-managing systems, e.g., WordPress [61], Joomla! [62], Magento [63] and Drupal [64]. JCH Optimize will enable editors to minimise the number of HTTP queries whereas GZIP and Minify will help reduce the size of the website, optimize images, support CDN, etc.

YSlow results showed that more than 50% of the websites fell below adequate levels. With reference to the five most frequent issues, we propose the reduction of HTTP query numbers using web optimising extensions. Furthermore, we recommend setting header visibility times, and a proper configuration of entity tags, i.e. ETags. (For Apache servers, with an addition to the.htaccess file.) The use of content delivery networks (CDN) to store multimedia contents is encouraged. Also, cookie-free servers should be used to serve static contents.

During the analysis of total website loading times, it was revealed that a quarter of the sites loaded within 6 s. In our point of view, loading times could be further reduced with the use of web optimisation (e.g. JCH Optimize). As for total website size, almost half of the sites exceeded 4 MB. This could be reduced with the use of proper applications. Within the last usability analysis, we had a look at the number of HTTP queries. Here, it can be stated that more than 50% of the websites had over 80 queries which further increased page loading times. We encourage the website operators to check their websites and try to reduce the number of individual website elements to be loaded, for example, by removing superfluous extensions, or minimising or merging HTML, CSS and JavaScript contents. Web optimising extensions can be of great help to carry this out.

5.2 Accessibility

In the course of the accessibility analysis, we scrutinised if the websites of public sector bodies complied with the WCAG 2.0 AA-level, the WAVE developer recommendations, and the type of accessibility. It was revealed by the WCAG analysis that every website included a certain number of faulty elements and thus, none of the websites could be regarded as accessible. Based on the five most frequent problems, we propose that the content creators should pay more attention to filling out additional fields when creating new content. Blind or partially sighted people usually use screen readers to access the screen content. These software tools read out the contents of the websites based on the source code. In order for the readers to be able to "read" the images, they must be provided with an alternative description, and in the same way, other non-textual elements must be accompanied by a textual description, an alternative name. One of our most important suggestions is to use the <alt> attribute when inserting a link (either text or an image [65]), which describes the functionality and/or target of the link. Additionally, provide labels to identify all form controls (text fields, checkboxes, radio buttons etc.). In most cases, this is done by using the <label> element [66], which describes the purpose of the form control. Finally, in cases where a button is used that contains an icon and no (visible) accompanying text (e.g. search button, hamburger menu button), it should always be provided with an attribute (e.g. aria-label [67]) that describes the function of the button. This is important so that users who use some assistive technologies can understand how the button works.

WAVE revealed that every website contained a certain number of warnings (lower-priority problems). Based on the five most frequent notifications, we firstly suggest that the content creators avoid the duplication of neighbouring links to contain the same element. Furthermore, developers ought to check the presence of header formatting—in case they are missing, they need to be introduced while having faulty URLs removed (in content manager systems, with the help of an installed extension). In addition, it should be checked whether the elements have the appropriate alternative text descriptions. According to the WAVE recommendations, there are three common problems (redundant title text, justified text, very small text), the first of which was pertinent to more than half of the websites. Based on these problems, we recommend that content creators should avoid such duplications where the title text matches the alternative text as it unnecessarily makes it slower to browse for users using a reading software. In most cases the title attribute can be removed, otherwise modify it to provide advisory, but not redundant information. Furthermore, the use of justify-type texts is not recommended as the spaces in between words can appear as “rivers of white” [68], which could cause reading problems for users with dyslexia. Our last suggestion is to avoid using too small font sizes because small text makes reading difficult for visually impaired people. We recommend that website use a minimum font size of 0.75 rem (12px). When checking the type of accessibility, it showed that more than half of the websites used segregated accessibility, and all but one included the icon of a yellow circle with three black dots on it to reach the accessible version. The negative effects of segregated accessibility have been discussed in the ‘related works’ section. Today’s modern web technologies enable us to ensure accessibility with considerably less effort than 10 years ago. Website owners are highly encouraged to leave behind the practice of segregated accessibility.

Accessibility problems encountered on CMS-based websites were classified into two groups. The first group includes so-called content formatting issues. This type of problem occurs when the user has used incorrect or incomplete formatting (e.g. justified formatting, use of a link without alt attribute) for the uploaded content. Using content formatting checkers and repair plugin (such as a11ychecker [69]) in WordPress [61] or Joomla! [62] may help solve these types of issues. This allows the user to verify and correct the used content formatting in the WYSIWYG editor [70] immediately after uploading the content. The second group includes website structural issues. However, it is important to note that unlike in static HTML pages, in CMS the structural elements of the website (main system, plugins) and the uploaded content are separated. As a result, the issues of page structure and content can be corrected differently. For problems of this type (e.g. a button appearing in a plugin but does not have an aria-label attribute), it is advisable to locate the faulty plugin and fix it. However, there is another option, which is to overwrite the source code of the website directly with a plugin (e.g. Real-Time Find and Replace [71] for WordPress, ReReplacer [72] for Joomla!) before displaying it in the browser. The plugin allows the user to specify a rule set which will replace the incorrect source code (e.g. aria-label attribute can be added to a button in directly the source code). After a bug fix, it is always advisable to check the result, for example an online accessibility checker (e.g. WAVE [48]).

5.3 Security

As part of the security analysis of the websites, we examined the proportion of web servers used, the web engines and programming languages alongside with the use of SSL encryption. When examining the servers of the websites, we concluded that almost all websites allowed us to query the web server type, and in most cases, even the exact version number. Since these are extremely sensitive pieces of information, which makes it possible for even a mediocre hacker to crack the system, it is especially important to keep the website’s server up-to-date, and to block information queries concerning its version [73]. As far as website engines are concerned, it turned out that less than 50% of the websites allowed queries for version numbers. Here, we recommend using the latest versions to increase not only security but also the speed [73]. Regarding the website engines, less than half of the websites had the content manager’s version numbers available, but when this information is accessible, it allows for a vulnerability window. We would like to emphasise the importance of having an up-to-date CMS and the fact that operators should always hide engine version numbers to reduce the risk of getting hacked [74,75,76]. When checking for SSL use, less than half of the sites provided secure communication, since without the use of SSL, sensitive information can be forwarded without encryption. An attacker could easily catch and extract this. Thus, the use of the SSL protocol is extremely crucial.

5.4 Language and GDPR

Finally, we examined in what languages are the websites of public sector bodies available, and to what extent they comply with GDPR regulations. As for the languages, it was found that more than half of the websites were available in at least one foreign language. Following this positive practice, we encourage public sector bodies to make the most relevant content and services available to people visiting the country in at least one foreign language. For CMS-based websites, there are basically two ways to support the multilingual contents on a website. The first is to integrate an automatic translation service (such as Google Translate) into the website, though in our experience the use of these services significantly reduces the quality of the translation. The second method is to use the internal language manager [77, 78] in the CMS. When creating new content in this system, it is possible to create and merge additional foreign language versions of that content. Changing content between languages is done automatically. Although the second method is manual translation, we believe that it provides better translation quality. In addition, using this method allows us to display different information in different languages, without having to wait for the results of the external translation service to load the page faster.

Examination of GDPR compliance revealed that no website could fulfil the requirements of the directive. One of the most important issues that was encountered on all websites was the incorrect display of cookie warnings. The cookie warnings were displayed in a simple pop-up window. However, the directive requires the website to provide a dedicated interface that lists all cookies on the website. This interface should allow the visitor to choose which cookies are allowed and which are not. We also recommend that all pages have a privacy and cookie description to further improve GDPR compliance. In addition, when using forms (e.g. contact forms) that request some personal information then this form must have a checkbox, which approves personal data management.

6 Conclusions

In this study our aim was twofold. First, a methodology has been created to ascertain the usability, accessibility, and security of websites. Second, this methodology has been applied to websites of Hungarian public sector bodies in order to check the quality of these websites in general, and in particular the compliance to the directive in [1]. The results indicate that there is plenty of room for improvement. We began by exploring relevant literature, looking for existing works to get to know the current state of art on this topic. After that, we tested various online tools which are capable to analyse the usability, accessibility, and security of websites. The most appropriate tools were selected, and a study of a total of 25 Hungarian websites of public sector bodies was conducted. The obtained results were exported to a central table, where they were organized by error types. The results were published on our referenced website [47], where all results in categories can be accessed in full detail. In the final phase of the study, we came up with proposals in order to improve the usability, accessibility, and security of these websites.