Public sector bodies are increasingly relying on the Internet. On this channel, indispensable information is transmitted to the public and a wide range of services is already available. Therefore, the usability, accessibility, and the security of these websites are very important. Accessibility is particularly crucial for persons with disabilities. The accessibility of public service websites is regulated by a number of laws; among others, the directive “on the accessibility of the websites and mobile applications of public sector bodies” adopted by the European Parliament in 2016. This obliges all European Union member states to make all public sector websites and mobile applications accessible by 23 September 2021. In practice, this means that websites must fulfil the level AA recommendations in WCAG 2.1. In our study, a website assessment method is developed by comparing different analytical tools. With this method, we analysed how Hungarian websites of public sector bodies fulfil the requirements of the directive. We have also investigated how well they comply with usability and security guidelines. The results showed that none of the 25 websites of the examined Hungarian public sector bodies could completely fulfil the recommendations of the Web Content Accessibility Guidelines (WCAG) and that half of the websites had only the lowest level of compliance in usability tests. From the security point of view, almost half of the websites use outdated server versions and programming language, which is very critical. We have proposed several suggestions to address the major problems, so website developers and administrators can improve the accessibility, usability, and security aspects of these websites.
Today the Internet is an essential way to access information and services. People use it for communication and getting information. The continuous online presence has resulted in the availability of more and more government services on the web following the private sector. So that people with disabilities can have easy access to these services, it is important that websites are easy to use, accessible, and properly secure. To fulfil these conditions, we have set out three main requirements that are strongly recommended to every website owner.
Usability is one of the most important requirements of websites, which also plays an important role in public websites. This indicator shows how easily and efficiently the website can be used to access the desired service. If a website is too slow or too complicated to manage, it increases user dissatisfaction and the risk of clicking away.
Another very important requirement is the accessibility of websites of public sector bodies, which allows persons with disabilities to perceive, understand, and interact with websites and their services. The European Union directive on accessibility of websites and mobile applications  obliges public sector bodies to comply with the requirements of WCAG 2.1 (Level AA) . Compliance with the requirements helps a wide range of people with disabilities, including blindness and low vision, deafness and hearing loss, learning disabilities, cognitive limitations, limited movement, speech disabilities, photosensitivity and combinations of these to gain access to websites and mobile applications of public sector bodies.
The third requirement is security, which is also essential, since users need to provide sensitive information to access each service. In the age of web content management systems, it is of utmost importance that communication is made through an encrypted channel, the system used on the website is up-to-date, and contains up-to-date extensions from a reliable source. If these basic security steps are missed, it will greatly increase the possibility of SQL injections, Cross-site scripting (XSS) and other website attacks, which may result in significant data loss.
The legislation passed in the European Parliament applies to all member states, including Hungary. According to the adopted directive, all websites and mobile applications of public sector bodies must be accessible by the end of 2021, otherwise the country may be penalized. In order to implement this process, a comprehensive study of the state of Hungarian government websites is required as a first step. Although some studies [3,4,5] on Hungarian websites were conducted in the early 2010s, the results of these studies are outdated and some of them did not focus specifically on websites of public sector bodies. The aim of our study was to carry out a survey providing up-to-date information on the Hungarian government websites and to develop suggestions for the problems that may arise.
The remainder of this paper is structured as follows: Sect. 2 provides a review of the relevant literature. Section 3 describes the methodology used in the study. Section 4 presents the results of the study. Section 5 makes suggestions to improve the issues presented in Sect. 4. Finally, Sect. 6 presents the conclusions.
European Union legislation
The providers of information and services, such as public sector bodies, rely increasingly on the internet in order to produce, collect, and provide a wide range of information and services online which are essential to the public. For this reason, it has become important to develop a directive that describes accessibility principles and techniques, that should be followed when designing, building, maintaining, and updating websites and mobile applications, in order to make them more accessible for users, in particular for persons with disabilities. On 22 December 2016, the European Union directive (2016/2102)  on the accessibility of websites and mobile applications of public sector bodies came into force, that provided persons with disabilities with better access to the websites and mobile applications of public sector bodies. Until further provision, the directive is based on the requirements of the harmonized European standard on Accessibility requirements for ICT products and services (EN 301 549) . Chapter 9 of the standard requires websites of Information Communications Technology (ICT) to fulfil all five requirements of WCAG 2.0 AA level . Starting with version 2.1.2, the standard requires WCAG 2.1 AA level . WCAG 2.1 includes all the eligibility criteria for 2.0 with 17 additional criteria  on mobile accessibility, people with low vision, people with cognitive and learning disabilities. As a result of this directive, all new websites of public sector bodies of EU Member states released after 23 September 2019 should be accessible, and after 23 September 2020, all previously created websites of public sector bodies should be accessible as well. After September 23, 2021, all mobile applications of public sector bodies must be accessible.
Usability test for public sector bodies websites
During our literature review we found various types of studies on websites of public sector bodies. One of the main types of the reviewed papers examined only the usability of the pages. Usability refers to how effective users are in using the site and how easily they access the content. Usability analysis basically uses two types of testing approaches: analysing all pages of a website or focusing only on the home page of a website. Home pages are often the most accessible pages of a site and developers usually pay attention to this. Numerous studies [10,11,12] have shown that there is a correlation between problems detected on a home page and other pages of the same website. This is especially true for content management systems, where the structure of the website is generally fixed and only the content is different. Most articles used automatic online tools. Stowers  defined six criteria to explore usability issues. These are the following: online services, user assistance, navigation, legitimacy, information architecture, accessibility. In , it has been shown what are the main factors that affect the usability of e-government websites. These include navigation, efficient readability, the use of images and graphics, multimedia technology, page structure, and the ability to search for information. Huang and Benyoucef  have defined 13 guidelines and 13 criteria to measure the usability and credibility levels of e-government sites. Choudrie and Weerakkody  have tested numerous e-government websites of several countries (Singapore, Finland, Canada, Hong Kong, Australia) with the WebXact tool. The analysis included testing faulty links, download times, elapsed time since the last update, style sheets, server-side image maps, internal multimedia elements, and metadata elements, browser compatibility issues, and HTML verification errors. The results showed that many governments ignore accessibility and quality criteria. Islam et al.  examined the usability of 6 e-government websites in Bangladesh with the help of 22 graduate students. The results showed that the tested websites had many critical usability issues, which need to be improved as soon as possible.
Accessibility test for public sector bodies websites
Accessibility and usability are closely related aspects in creating websites that work for everyone. Their goals, approaches, and guidelines overlap significantly. Web accessibility means that people with disabilities can equally perceive, understand, navigate, and interact with websites. The EU directive  also applies to this. Like usability analysis, accessibility analysis uses two types of testing approaches: analysing all pages of a website or focusing only on the home page of a website. There are several methods to check the accessibility of a website, one of these is the use of online checkers. Abanumy et al.  studied the e-government pages of Saudi Arabia and Oman with various online tools. Among other, the Multiweb, the LYNX, and the W3C checkers were used to examine how websites interact with different input devices, the support of text-based browsing, and the presence of HTML syntax problems. Shi  discovered a number of major accessibility issues in the Chinese e-government websites, and some minor bugs in the Australian government websites. A year later , Shi analysed 324 local Chinese local government websites, many of which repeatedly violated the W3C accessibility guidelines. Many similar surveys  were conducted in Europe (England, France, Germany, Switzerland), Asia (China, India, Cambodia, Philippines) and finally in Africa (South Africa, Libya, Namibia, Kenya). The results show that most of the tested websites did not comply with WCAG 1.0 verification standards. Kopackova et al.  have analysed 39 websites of Czech public sector bodies based on civic aspects and search engine discovery. They found that the government websites contain deficiencies, which make it difficult for users to find and display the information they need. Al-Khalifa  examined 36 Saudi Arabian websites of public sector bodies based on accessibility requirements. As a result of the research, none of the websites were found able to complete the WCAG 2.0 conformance test in 2010. Research has shown that a high percentage of website errors are level A errors, and serious efforts should be made to ensure that websites fulfil the minimum requirements. Al-Khalifa et al.  in 2017 re-examined the Saudi Arabian websites of public sector bodies. Analysis has shown that over the past 5 years, the number of violations of WCAG 2.0 rules has been drastically reduced, and more and more websites have supported mobile appearance.
Hungarian unique practice, the segregated web accessibility
Unfortunately, segregation due to disability still exists today. When people with disabilities are asked to use a separate interface to access the information on a given website, it is basically one type of discrimination. Many studies have already been conducted on the negative effects of segregated accessibility. Lazar et al.  have explained that segregated solutions typically navigate disabled people with limited functionality and less up-to-date information. A typical example of this practice is shown in Figs. 5 and 6 in the “Appendix”. According to Lazar, modern programming techniques allow creators to design websites that are accessible to everyone without segregation. The designation of segregated web accessibility is unique in Hungary. In this practice, the accessible version of a website is indicated by three black dots on the yellow circle as shown in Fig. 1. This designation is currently being used in several countries to support people with disabilities (e.g. at road signs), but as far as we know, regarding websites, it is used only in Hungary. However, an “accessible version” of such websites usually only concentrates on the contrast only, it has black background and increased yellow letters. But on the other hand, these pages mostly have limited functionality, they are rarely updated, and do not fulfil the WCAG requirements. In our view, this bad practice may also have contributed to the lack of accessibility of Hungarian websites.
Website security tests
In addition to usability and availability, the third aspect we consider important is security. Although security is a bit farther from the previous two aspects, the security of websites is at least as important as usability and accessibility. A Content Management System (CMS)  based website is basically considered secure if the communication is via an encrypted channel, the data is properly encrypted, and the parts of the website (base system, installed extensions) are up to date. Website security is an important topic, so many publications have been made on this. The Open Web Application Security Project (OWASP) has defined  the ten most serious security threats to web applications. Choudrie et al.  have investigated data protection using the WebXact tool on websites of public sector bodies in many countries. The results showed that no portal used encryption. In this study, it has been shown that 81.6% of 212 e-government websites contain some XSS and SQL injection vulnerabilities . Shteiman  described the importance of upgrading CMS. Nowadays, content management systems have become very popular , therefore they have become popular targets for hackers. Many website owners do not keep up to date with their CMS and installed extensions, making them vulnerable to various hacker attacks due to public vulnerabilities. Abu-Shanab et al.  have reviewed Jordan's e-government websites using the Rapid7 monitoring tool. Studies have shown the websites under investigation contain a number of vulnerabilities that require a quick solution. Sahu et al.  have discussed the issue of safe web application development. It is recommended that more secure web applications should be created in compliance with the Secure Coding Standards.
Comprehensive website tests based on accessibility, availability, and security
The three aspects outlined above are important in themselves, but in a comprehensive study it is advantageous to consider them together. There are studies that analyse websites in specific countries based on accessibility, availability, and security. Ismailova  has analysed 60 Kyrgyz e-government websites using online diagnostic tools based on usability, accessibility, and security considerations. The survey showed that 46% of websites contained some usability issues and 69% had some accessibility issues. In the final phase of the study, it turned out that 55% of Kyrgyz websites used an outdated version of a content management system that contained public vulnerabilities. Noe  examined 79 Tanzanian e-government websites for usability, accessibility, and security aspects. The analysis used automated diagnostic tools such as Pingdom, Google Speed Insight, WAVE, W3C Checker, and Acunetix. The results showed that 100% of the websites had broken links in terms of usability, and in 52 out of 79 websites, it took more than 5 s to load. Accessibility results showed that none of the web sites were able to fulfil the WCAG 1.0 policies. The results of the security analysis revealed that 40 of the 79 websites tested had one or more high-severity vulnerabilities (SQL injection, Cross site scripting). Based on the results, the study makes some recommendations to improve the usability, availability and security of public institutions in Tanzania. Arvinder et al.  have analysed 280 hospital websites in many Indian cities for accessibility, accessibility, and security. Data were collected using automated tools (WebSiteOptimization, Measure Text Readability, TAW) for the survey. Accessibility analysis showed that 280 pages had a total of 22,491 errors. The minimum error was 3 and the maximum was 840. In the first part of the usability tests, the authors examined the readability of hospital websites. Websites were ranked on a scale of 0–100, where higher values meant better readability. Pages received an average of 46.7 of the Flesch-Kincaid readability score, which was the “Difficult to read” level. In the second phase of usability tests, many page parameters (page size, number of objects, total image size, page load time) were examined. The results showed that websites have an average size of more than 20 MB and contain an average of 117 objects, which significantly increased the page load speed. A security analysis revealed that 70 of the 280 websites examined were CMS-based. Some 25% of them had security issues due to non-upgraded software components.
Previous studies [3,4,5] have shown that more than 90% of Hungarian websites are not accessible. Due to the limited implementation period of the EU Directive, new information was needed to determine the current national outlook. At the outset of the study, the assumption was that the websites of public sector bodies have not improved significantly from previous investigations.
The aim of the study is to test the most important Hungarian public sector websites. Hungary is divided into 19 counties. The websites of these county centres, 5 cities with county rights and the website of the Government of Hungary were investigated. Although the counties are subdivided into a total of 174 districts, their detailed examination goes beyond the scope of the current article. This study specifically focuses on the home page as a metric for web accessibility in general.
There are various web usability, accessibility, and security monitoring tools available on the Internet [36,37,38]. In the preparatory phase of the study, the selection of the appropriate tools was carried out based on various tests and evaluations. After that, with the help of the selected tools, we carried out a complete examination of 25 websites of Hungarian public sector bodies.
Evaluation of websites of Hungarian public sector bodies from usability aspects
Usability has various definitions. Some define usability in a literal sense, how logical the website is, how easy to find the information, how easy to navigate between the pages. The testing of this type of usability is difficult, because the definition is subjective, and the test can only be performed manually. We use another definition of usability, which focuses on measurable properties like heatmap , scrollmap , page load speed, etc. In this study, we focused on the page load speed analytics, because it correlates well with user retentions. Many tools were tested and according to our opinion GTmetrix is the most versatile one. GTmetrix  is a usability testing tool concerned with page speed analytics, which is based on various web performance optimization APIs (Google PageSpeed , Yahoo YSlow ), as well as parameters that affect usability (e.g. fully load time, total page size, number of HTTP requests). PageSpeed and YSlow are displayed on a scale from A to F, which is evaluated based on the classification of the corresponding rule of the optimization tool. After each test, GTmetrix displayed the score of the website and the percentage of each rule and compares them with the average values.
During the measurement, the test environment was set based on different parameters:
Test location The test computer (London, UK) closest to the Hungarian servers was selected;
Browser The most commonly used browser Google Chrome (62.0.3202.94), has been selected;
Connection Since broadband coverage in Hungary is adequate and the speed of mobile internet is at the forefront of the world , no restrictive parameters have been set for speed, so the unthrottled connection on the twisted pair was selected;
Versions of web performance optimization tool PageSpeed 1.15-gt1, YSlow 3.1.8.
The GTmetrix tool provides the following information:
PageSpeed Score Specifies the percentage of PageSpeed recommendations that a website can fulfil ;
YSlow Score Specifies the percentage of YSlow recommendations that a website can fulfill ;
Fully loaded time (s) Specifies the total number of seconds which is needed for the page content to load. Google recommends loading in up to 2–3 s in one study . In our study, the average page load time of the websites checked by GTmetrix during the last 30 days was 7.2 s;
Total page size Page size also increases the page load time. In our study, the average page size of the websites that GTmetrix checked for the last 30 days was 3.1 MB;
Requests Specifies the number of HTTP requests for the page. The number of HTTP requests on a page is also directly proportional to the page load time. In our study, the average number of HTTP requests for websites checked with GTmetrix in the last 30 days was 88.
After the test environment was set up, all 25 websites of public sector bodies were diagnosed based on availability one by one. During the analysis, the results were manually entered into the central table grouped by error type .
Using GTmetrix is straightforward. In order to begin the investigation, the name of the website to be verified must be provided on the website of GTmetrix. After that GTmetrix displays the results in tabular form, as shown in Fig. 2. The Performance Scores window displays the cumulative evaluation of the API being tested, and the Page Details window displays a general summary (Fully Loaded Time, Total Page Size, Request). At the bottom of the page is a list of suggestions that are tested for each API.
Evaluation of websites of Hungarian public sector bodies: accessibility aspects
There are a number of tools for checking the accessibility of a website. One of these, in fact the most flexible, is WAVE  online checker. WAVE is developed by the WebAIM , a non-profit organization. Due to the continuous development of the software, more and more WCAG 2.1  rules can be verified. Another advantage of this tool is that it graphically displays those website elements which do not pass the accessibility standards. These problems are grouped into categories, and after clicking on the error, it can be seen which rule has been violated, and it gives a tip for solution.
Since 2019, WAVE has been able to validate websites up to WCAG 2.1 AAA levels, but we used an earlier version of WAVE in our study that had the following levels of compliance:
WCAG 2.0 A (basic level of the Web Accessibility Guidelines);
WCAG 2.0 AA (the second level of the Web Accessibility Guidelines, which includes level A compliance);
Full (WCAG 2.0 AA + recommendations of WAVE developers).
In the course of the study, we made accessibility checks on each of the 25 websites of public sector bodies. We used the Chrome plugin version of WAVE Evaluation Tool (1.0.9) with Full compliance test during the scan. At the time of analysis, the results were manually entered into the central table and grouped by error type .
WAVE is simple to use. Once installed, a green WAVE button will appear on the upper right side of the browser. This button must be pressed after loading a website. As shown in Fig. 3, WAVE summarizes the detected problems on the left side of the screen, while on the right shows these problems graphically. By pressing the <code> button at the bottom of the website, the source code of the website is displayed. After clicking on an error, WAVE details the problem and then jumps to the HTML source code to be repaired in the source code window. Based on this information, the repair of the page under review can begin.
In the final phase of the analysis, for all websites of public sector bodies we investigated if the site uses an integrated or a segregated accessibility solution. We also checked whether the website contains an icon for the accessible version of the website, as has become a practice in Hungary (marked with three black dots on a yellow circle).
Evaluation of websites of Hungarian public sector bodies: security aspects
There are many web-based security monitoring software tools available on the Internet. A couple of such devices have been tested and the Sucuri  website inspector, a subsidiary of the American GoDaddy hosting company, was selected. In our view, this tool can retrieve the details of the websites in sufficient detail to help determine whether or not a particular website represents a security risk for users.
The Sucuri tool provides the following information:
Using TLS (SSL);
Web server type and version;
Website engine and version in use;
Programming system and version in use.
In the course of the survey, security checks were carried out one by one on all 25 websites of public sector bodies. During the analysis, the results were manually entered into the central table and grouped by problem type .
To start an experiment, the name of the page to be scanned must be entered. Figure 4 shows that after the check, Sucuri displays the results in different blocks of information. The top block contains the usual information on the website (IP address, Hosting, Server, CMS etc.). Figure 4 shows a graphical representation of the security rating of the website. Below further test results can be found (Website Malware & Security, Website Blacklist Status, Hardening Improvements).
Evaluation of websites of Hungarian public sector bodies: other aspects
We used GDPR verifier  of the Danish Secure Privacy web security company to examine certain parameters of GDPR compliance  for websites of public sector bodies. The cookie compliance defined in the directive could only be checked manually because of its complexity.
The Secure Privacy tool provides the following information:
Prior consent on other than strictly necessary cookies (ePrivacy);
Prior consent on personal data (ePrivacy);
Personal data is transmitted to 'adequate countries' (GDPR);
As a member of the European Union, it is important for Hungary to support the publication of content in several languages. As there were no tools available for automated evaluation of the languages available on the site under review, multilingual analysis was performed manually for all 25 public websites.
Limitation of online testing tools
The use of free automated testing tools is quite attractive as it is cost effective, easy to use, easy to access, and provides fast results. However, such tools have a number of limitations. Automated testing programs may mistakenly identify items as accessible or inaccessible, disregard the different disabilities or abilities of people with similar disabilities, fail to address usability or functionality issues, and omit accessibility issues that can only be identified by humans (keyboard access, meaningful alt texts). Automated software has an average error rate of at least 30%, depending on the test tool. Using automated tools can give the false impression that a good rating results in a completely accessible website . While automated tools are not a complete solution, they are a necessary part of any serious accessibility evaluation effort.
For usability, we used the GTmetrix online checker. Pages submitted by users for analysing are processed in a queue, which may take some time. The tool focuses specifically on the page load time, so it cannot perform heatmap or scrollmap tests. A further limitation is that subscribers are required to use more detailed settings (device simulation, screen resolution, custom connection, etc.) when testing.
For accessibility, we used the WAVE online checker. WAVE's limitations include being able to test only one page at a time, not the entire website and the results are not exportable. The tool cannot check all of the issues in WCAG 2.1 guideline—no automated tool can.
For security, we used the Sucuri online checker. It may give a false sense of security when device indicates a low or minimal security risk because the version number of extensions installed cannot be verified when using a content management system. The device can only analyse one page at a time. Results cannot be exported.
In this study, 25 websites of public sector bodies were examined, which include the government websites of the county seats and the most important districts of Hungary and the website of the Hungarian Government. A detailed list of the websites in the examination is given in of the “Appendix”. In addition to usability, accessibility, and security analysis, support for GDPR compliance and multilingualism has also been investigated. The results of each examination were systematized and then classified according to usability, accessibility, and security.
Analysis of PageSpeed
In the first phase of the usability study, PageSpeed results were analysed. Table 1 shows how many websites have passed the PageSpeed analytics rules at different levels. An “A” rating means that the website has completed the PageSpeed recommendation fully or almost fully. The investigation revealed that only one website was able to achieve the highest rating, followed by three, and then the subsequent three sites achieved average results. More than half of the websites have reached the worst level of compliance.
Analysis of YSlow
In the next phase of the study, we analysed the results of YSlow. Table 3 shows how many websites have passed the rules of YSlow analytics. Rating “A” means that the website has completely or almost completely fulfilled the YSlow recommendations. The investigation revealed that none of the websites could achieve the highest rating. Out of 25 websites examined, 19 were below average.
The 25 websites failed 10 different YSlow analytical rules fully or partly, with the 5 most common problems listed in Table 4. Based on the survey, public websites of public sector bodies do not use content delivery networks (CDN) to store multimedia content. The majority of websites does not use CDN server for static contents, and cached content was not set to expire. Over half of the websites have over-frequent HTTP queries, and entity tags (ETags) configuration is missing without which cannot be ensured that the resource in browser cache and on the web server are identical.
Analysis of fully loaded time
In the following analysis, the total load time of the websites was examined. The results are shown in Table 5. The study revealed that 9 websites achieved excellent results with a load time of 3 s, followed by 9 results achieving good ranking. For the last 7 websites, users have to wait more than 6 s to load the website, which significantly increases the risk of clicking away.
Analysis of total page size
In the following analysis, we analysed the total size of the websites, the classification of which is shown in Table 6. The study revealed that 9 websites achieved excellent results with a load time of 3 s, followed by 9 results. For the last 7 websites, users need to wait more than 6 s to load the website, which increases the risk of clicking away.
Analysis of number of HTTP requests
In the last section, we examine the number of HTTP requests on websites, which are listed in Table 7. Based on the results, it can be said that about half of the websites work with fewer than 80 HTTP requests. Out of the 25 websites examined, 7 have an average of 80 and 119 requests and 7 websites have to wait for more than 120 HTTP requests while browsing the government websites.
In the first phase of the accessibility study, the WCAG 2.0 AA level compliance required by the directive was analysed. Table 8 shows that all of websites of the study contain at least one faulty item, so it can be said that none of them are fully accessible. We have found with the help of the WAVE results that more than half of the websites had relatively few errors, but 8 websites contained more than 30 faulty items.
The investigated websites ignored several WCAG 2.0 AA requirements. The five most common errors are shown in Table 9. Based on the survey, almost all websites contained empty links (does not describe the functionality and/or target of that link), more than half of the websites contained images that did not have alternative text, contained a form element without item description, or a button without description.
Table 10 shows the classification of the warnings received as a result of the WAVE verification. The results indicate that almost every website contains a warning. More than half of the websites had relatively few warnings, but 6 websites contained more than 30 faulty items.
The five most common warnings received during the study are shown in Table 11. Based on the results, it turned out that almost all of the websites contained a link that is also reffered to by an adjacent element and a little less than half of the websites did not have an <h1> element and a heading level is skipped (e.g. an <h1> is directly followed by an <h3>.
Compliance with recommendations of WAVE developers
In the next phase of the study, we have analysed accessibility compliance with the recommendations proposed by the WAVE developers. Table 12 shows that for more than half of the tested websites, the title attribute of a HTML link was identical to the text of the link. This unnecessarily slows down the browsing of the page by users who use screen reader software. In addition, 5 pages used justified format, which causes reading difficulties for people with dyslexia, and 2 pages used 10px or less font size that causes reading difficulties for people who have mild vision impairment.
Type of accessibility
In the last phase of the study, we have assessed the type of accessibility and the use of the icon navigating to the accessible version of the website. The study found that 16 out of the 25 tested websites were using Hungarian unique practice, the segregated web accessibility. As explained in more detail in the third paragraph of Sect. 3, it means that these websites have two versions, but unfortunately the accessible version most of the time is not updated and has less functionality. With the exception of one website, all segregated accessibility websites are indicated by three black dot-based icons on the yellow circle to navigate to the accessible version of the website.
During the security investigation, it was found that on the websites where the version of the web server, the engine of the website or the type of programming language were accessible, the versions that were in use contained an already known vulnerability.
In the first phase of the security investigation the distribution of web servers was measured. Table 13 shows that 23 out of the 25 public sector sites were able to query the web server type, 13 of which had the exact version number, all of which needed updating.
Table 14 shows that the availability of the types of website engines was a little better, as only 8 of the 25 websites had their engines and versions publicly accessible. For websites where the version number is available, the engine contains an already known vulnerability that greatly increases the risk of hacking.
For the programming languages, 11 of the 25 versions can be retrieved, each with a public vulnerability. The available version numbers all contained PHP programming language.
In the last phase of the study, the use of SSL websites was investigated. Our study revealed that only 10 of the 25 public websites used SSL encryption.
Analysis of other aspects
In this study, we used the Secure Privacy  tool to assess the compliance of GDPR directive  with the 25 public websites. The survey revealed that none of the tested websites fulfils the requirements completely.
It was also a common problem that none of the forms on the websites had a check box that would allow the site administrators to manage the data entered on the form.
The pages have been tested for multilingualism, that 16 of the 25 websites contained at least one additional language. Table 15 also shows that more than half of the websites had an English version and in 9 cases also had a German version.
Based on the results we have developed several suggestions for improvement, which are presented in the following subsections. Since more than 57%  of websites on the Internet are CMS-based (and this number is constantly increasing), it is worthwhile to repeat the checks at least every 6 months, and when the main version of the CMS is changed, or when extensions are updated. Validation can be assisted by automated validation tools (which we have also introduced), but it is important to note that none of them can completely replace manual validation.
Based on our own experience, most of the problems can be traced back to the fact that governments are not fully aware of their legal obligations with regard to their websites (e.g. EU directives). Furthermore, they cannot provide sufficient resources to educate their content editors. The solution could be a national or EU partnership, where funding resources could be used to develop the necessary new methods and educate and train people. It is positive that many research and pilot projects [55,56,57,58] are currently underway with this objective.
YSlow results showed that more than 50% of the websites fell below adequate levels. With reference to the five most frequent issues, we propose the reduction of HTTP query numbers using web optimising extensions. Furthermore, we recommend setting header visibility times, and a proper configuration of entity tags, i.e. ETags. (For Apache servers, with an addition to the.htaccess file.) The use of content delivery networks (CDN) to store multimedia contents is encouraged. Also, cookie-free servers should be used to serve static contents.
In the course of the accessibility analysis, we scrutinised if the websites of public sector bodies complied with the WCAG 2.0 AA-level, the WAVE developer recommendations, and the type of accessibility. It was revealed by the WCAG analysis that every website included a certain number of faulty elements and thus, none of the websites could be regarded as accessible. Based on the five most frequent problems, we propose that the content creators should pay more attention to filling out additional fields when creating new content. Blind or partially sighted people usually use screen readers to access the screen content. These software tools read out the contents of the websites based on the source code. In order for the readers to be able to "read" the images, they must be provided with an alternative description, and in the same way, other non-textual elements must be accompanied by a textual description, an alternative name. One of our most important suggestions is to use the <alt> attribute when inserting a link (either text or an image ), which describes the functionality and/or target of the link. Additionally, provide labels to identify all form controls (text fields, checkboxes, radio buttons etc.). In most cases, this is done by using the <label> element , which describes the purpose of the form control. Finally, in cases where a button is used that contains an icon and no (visible) accompanying text (e.g. search button, hamburger menu button), it should always be provided with an attribute (e.g. aria-label ) that describes the function of the button. This is important so that users who use some assistive technologies can understand how the button works.
WAVE revealed that every website contained a certain number of warnings (lower-priority problems). Based on the five most frequent notifications, we firstly suggest that the content creators avoid the duplication of neighbouring links to contain the same element. Furthermore, developers ought to check the presence of header formatting—in case they are missing, they need to be introduced while having faulty URLs removed (in content manager systems, with the help of an installed extension). In addition, it should be checked whether the elements have the appropriate alternative text descriptions. According to the WAVE recommendations, there are three common problems (redundant title text, justified text, very small text), the first of which was pertinent to more than half of the websites. Based on these problems, we recommend that content creators should avoid such duplications where the title text matches the alternative text as it unnecessarily makes it slower to browse for users using a reading software. In most cases the title attribute can be removed, otherwise modify it to provide advisory, but not redundant information. Furthermore, the use of justify-type texts is not recommended as the spaces in between words can appear as “rivers of white” , which could cause reading problems for users with dyslexia. Our last suggestion is to avoid using too small font sizes because small text makes reading difficult for visually impaired people. We recommend that website use a minimum font size of 0.75 rem (12px). When checking the type of accessibility, it showed that more than half of the websites used segregated accessibility, and all but one included the icon of a yellow circle with three black dots on it to reach the accessible version. The negative effects of segregated accessibility have been discussed in the ‘related works’ section. Today’s modern web technologies enable us to ensure accessibility with considerably less effort than 10 years ago. Website owners are highly encouraged to leave behind the practice of segregated accessibility.
Accessibility problems encountered on CMS-based websites were classified into two groups. The first group includes so-called content formatting issues. This type of problem occurs when the user has used incorrect or incomplete formatting (e.g. justified formatting, use of a link without alt attribute) for the uploaded content. Using content formatting checkers and repair plugin (such as a11ychecker ) in WordPress  or Joomla!  may help solve these types of issues. This allows the user to verify and correct the used content formatting in the WYSIWYG editor  immediately after uploading the content. The second group includes website structural issues. However, it is important to note that unlike in static HTML pages, in CMS the structural elements of the website (main system, plugins) and the uploaded content are separated. As a result, the issues of page structure and content can be corrected differently. For problems of this type (e.g. a button appearing in a plugin but does not have an aria-label attribute), it is advisable to locate the faulty plugin and fix it. However, there is another option, which is to overwrite the source code of the website directly with a plugin (e.g. Real-Time Find and Replace  for WordPress, ReReplacer  for Joomla!) before displaying it in the browser. The plugin allows the user to specify a rule set which will replace the incorrect source code (e.g. aria-label attribute can be added to a button in directly the source code). After a bug fix, it is always advisable to check the result, for example an online accessibility checker (e.g. WAVE ).
As part of the security analysis of the websites, we examined the proportion of web servers used, the web engines and programming languages alongside with the use of SSL encryption. When examining the servers of the websites, we concluded that almost all websites allowed us to query the web server type, and in most cases, even the exact version number. Since these are extremely sensitive pieces of information, which makes it possible for even a mediocre hacker to crack the system, it is especially important to keep the website’s server up-to-date, and to block information queries concerning its version . As far as website engines are concerned, it turned out that less than 50% of the websites allowed queries for version numbers. Here, we recommend using the latest versions to increase not only security but also the speed . Regarding the website engines, less than half of the websites had the content manager’s version numbers available, but when this information is accessible, it allows for a vulnerability window. We would like to emphasise the importance of having an up-to-date CMS and the fact that operators should always hide engine version numbers to reduce the risk of getting hacked [74,75,76]. When checking for SSL use, less than half of the sites provided secure communication, since without the use of SSL, sensitive information can be forwarded without encryption. An attacker could easily catch and extract this. Thus, the use of the SSL protocol is extremely crucial.
Language and GDPR
Finally, we examined in what languages are the websites of public sector bodies available, and to what extent they comply with GDPR regulations. As for the languages, it was found that more than half of the websites were available in at least one foreign language. Following this positive practice, we encourage public sector bodies to make the most relevant content and services available to people visiting the country in at least one foreign language. For CMS-based websites, there are basically two ways to support the multilingual contents on a website. The first is to integrate an automatic translation service (such as Google Translate) into the website, though in our experience the use of these services significantly reduces the quality of the translation. The second method is to use the internal language manager [77, 78] in the CMS. When creating new content in this system, it is possible to create and merge additional foreign language versions of that content. Changing content between languages is done automatically. Although the second method is manual translation, we believe that it provides better translation quality. In addition, using this method allows us to display different information in different languages, without having to wait for the results of the external translation service to load the page faster.
Examination of GDPR compliance revealed that no website could fulfil the requirements of the directive. One of the most important issues that was encountered on all websites was the incorrect display of cookie warnings. The cookie warnings were displayed in a simple pop-up window. However, the directive requires the website to provide a dedicated interface that lists all cookies on the website. This interface should allow the visitor to choose which cookies are allowed and which are not. We also recommend that all pages have a privacy and cookie description to further improve GDPR compliance. In addition, when using forms (e.g. contact forms) that request some personal information then this form must have a checkbox, which approves personal data management.
In this study our aim was twofold. First, a methodology has been created to ascertain the usability, accessibility, and security of websites. Second, this methodology has been applied to websites of Hungarian public sector bodies in order to check the quality of these websites in general, and in particular the compliance to the directive in . The results indicate that there is plenty of room for improvement. We began by exploring relevant literature, looking for existing works to get to know the current state of art on this topic. After that, we tested various online tools which are capable to analyse the usability, accessibility, and security of websites. The most appropriate tools were selected, and a study of a total of 25 Hungarian websites of public sector bodies was conducted. The obtained results were exported to a central table, where they were organized by error types. The results were published on our referenced website , where all results in categories can be accessed in full detail. In the final phase of the study, we came up with proposals in order to improve the usability, accessibility, and security of these websites.
European Union: DIRECTIVE (EU) 2016/2102 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. Off. J. Eur. Union (2016)
W3C: Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/. Accessed 4 Dec 2019
Merkovity, N.: Hungarian party websites and parliamentary elections. Cent. Eur. J. Commun. 4(7), 209–225 (2011)
Szeróvay, K.: Usability of e-Government websites, evaluation of the Hungarian e-Government portal. In: COFOLA 2011, pp. 1596–1635 (2011)
Lanyi, C.S., Czank, N., Sik, A.: Testing the accessibility of websites. Int. J. Knowl. Web Intell. 2(1), 87 (2011)
ETSI: EN 301 549 (V2.1.2): accessibility requirements for ICT products and services. https://www.etsi.org/deliver/etsi_en/301500_301599/301549/02.01.02_60/en_301549v020102p.pdf (2018). Accessed 26 Nov 2018
W3C: Web Content Accessibility Guidelines (WCAG) 2.0. https://www.w3.org/TR/WCAG20/. Accessed 19 June 2019
W3C: WCAG 2.1 adopted in European standard EN 301 549 for ICT. https://www.w3.org/WAI/news/2018-09-13/WCAG-21-EN301549/ (2018). Accessed 26 Nov 2018
W3C: What’s New in WCAG 2.1. https://www.w3.org/WAI/standards-guidelines/wcag/new-in-21/. Accessed 4 Dec 2019
Nielsen, J.: Designing Web Usability. New Riders Publishing, Indianapolis (2000)
Yu, H.: Web accessibility and the law: recommendations for implementation. Libr. Hi Tech 20(4), 406–419 (2002)
Loiacono, E.T., McCoy, S.: Website accessibility: a cross-sector comparison. Univers. Access Inf. Soc. 4(4), 393–399 (2006)
Stowers, G.N.L.: The state of federal websites: the pursuit of excellence (2002)
Byun, D.H., Finnie, G.: Evaluating usability, user satisfaction and intention to revisit for successful e-government websites. Electron. Gov. an Int. J. 8(1), 1 (2011)
Huang, Z., Benyoucef, M.: Usability and credibility of e-government websites. Gov. Inf. Q. 31(4), 584–595 (2014)
Choudrie, J., Weerakkody, G.G.V.: Evaluating global e-government sites: a view using web diagnostic tools. Electron. J. e-Government (2004)
Islam, M.N., Rahman, S.M.A., Islam, M.S.: Assessing the usability of e-government websites of Bangladesh. In: 2017 International Conference on Electrical, Computer and Communication Engineering (ECCE), pp. 875–880 (2017)
Abanumy, A., Al-Badi, A., Mayhew, P.: e-Government website accessibility: in-depth evaluation of Saudi Arabia and Oman. Electron. J. e-Government 3(3), 149–156 (2005)
Shi, Y.: E-Government web site accessibility in Australia and China. Soc. Sci. Comput. Rev. 24(3), 378–385 (2006)
Shi, Y.: The accessibility of Chinese local government Web sites: an exploratory study. Gov. Inf. Q. 24(2), 377–403 (2007)
Kuzma, K., Yen, J., Oestreicher, D.: Global e-government web accessibility: an empirical examination of EU, Asian and African sites. In: 2nd International Conference on Information and Communication Technology and Accessibility (2009)
Kopackova, H., Michalek, K., Cejna, K.: Accessibility and findability of local e-government websites in the Czech Republic. Univ. Access Inf. Soc. 9(1), 51–61 (2010)
Al-Khalifa, H.S.: The accessibility of Saudi Arabia government Web sites: an exploratory study. Univ. Access Inf. Soc. 11(2), 201–210 (2012)
Al-Khalifa, H.S., Baazeem, I., Alamer, R.: Revisiting the accessibility of Saudi Arabia government websites. Univ. Access Inf. Soc. 16(4), 1027–1039 (2017)
Lazar, J., Wentz, B.: Separate but unequal: web interfaces for people with disabilities. User Exp. Mag. 10, 12–13 (2011)
Wikipedia: Content management system. https://wikipedia.org/wiki/Content_management_system. Accessed 23 Aug 2019
OWASP: Open Web Application Security Project (OWASP): OWASP Top Ten Project. https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project. Accessed 26 Nov 2018
Moen, V., Klingsheim, A.N., Simonsen, K.I.F., Hole, K.J.: Vulnerabilities in e-governments. Int. J. Electron. Secur. Digit. Forensics 1(1), 89 (2007)
Shteiman, B.: Why CMS platforms are breeding security vulnerabilities. Netw. Secur. 2014(1), 7–9 (2014)
W3Techs: Usage of content management systems for websites. Web Technology Surveys. https://w3techs.com/technologies/overview/content_management/all (2017). Accessed 26 Nov 2018
Alsmadi, I., Shanab, E.A.: E-government website security concerns and citizens’ adoption. Electron. Gov. an Int. J. 12(3), 243 (2016)
Sahu, D.R., Tomar, D.S.: Analysis of web application code vulnerabilities using secure coding standards. Arab. J. Sci. Eng. 42(2), 885–895 (2017)
Ismailova, R.: Web site accessibility, usability and security: a survey of government web sites in Kyrgyz Republic. Univ. Access Inf. Soc. 16(1), 257–264 (2017)
Elisa, N.: Usability, accessibility and web security assessment of e-government websites in Tanzania. Int. J. Comput. Appl. 164(5), 42–48 (2017)
Kaur, A., Dani, D., Agrawal, G.: Evaluating the accessibility, usability and security of Hospitals websites: an exploratory study. In: 2017 7th International Conference on Cloud Computing, Data Science & Engineering—Confluence, pp. 674–680 (2017)
The Good Group Inc.: Top usability testing tools: the complete list (84 tools). https://thegood.com/guide/usability-testing-tools/. Accessed 26 Nov 2018
W3C: Web Accessibility Evaluation Tools List. https://www.w3.org/WAI/ER/tools/. Accessed 26 Nov 2018
OWASP: Open Web Application Security Project (OWASP): Vulnerability Scanning Tools. https://www.owasp.org/index.php/Category:Vulnerability_Scanning_Tools. Accessed 26 Nov 2018
Hergul, S.: Understanding simple heat maps for smarter UI design. https://www.uxpin.com/studio/blog/understanding-simple-heat-maps-smarter-ui-design/ (2015). Accessed 28 Aug 2019
Dossetto, F.: Scroll maps: 5 ways to optimize UX and increase conversions. https://www.hotjar.com/blog/scroll-maps/ (2019). Accessed 28 Aug 2019
GT.net: GTmetrix. https://gtmetrix.com. Accessed 26 Nov 2018
Google: Analyze and optimize your website with PageSpeed tools. https://developers.google.com/speed/. Accessed 26 Nov 2018
Yahoo: YSlow. https://yslow.org. Accessed 26 Nov 2018
Ookla: Speedtest Hungary Index. https://www.speedtest.net/global-index/hungary. Accessed 26 Nov 2018
GTmetrix: PageSpeed and YSlow recommendations. https://gtmetrix.com/recommendations.html. Accessed 16 Jan 2019
DoubleClick: The need for mobile speed. https://storage.googleapis.com/doubleclick-prod/documents/The_Need_for_Mobile_Speed_-_FINAL.pdf (2016). Accessed 10 Dec 2018
Csontos, B., Heckl, I.: Report of the Hungarian Governments websites 2018. https://dcs.uni-pannon.hu/files/docs/users/csontosb/report-of-the-hungarian-governments-websites-2018.xlsx (2018). Accessed 20 Nov 2018
WebAIM: WAVE: web accessibility evaluation tool. https://wave.webaim.org. Accessed 11 Dec 2019
Center for Persons with Disabilities: WebAIM. https://webaim.org. Accessed 26 Nov 2018
Sucuri: Free website malware and security scanner. https://sitecheck.sucuri.net. Accessed 26 Nov 2018
Secure Privacy: GDPR compliance checker. https://secureprivacy.ai. Accessed 26 Nov 2018
European Union: REGULATION (EU) 2016/679 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing. Off. J. Eur. Union (2016)
Jaeger, P.T.: Assessing Section 508 compliance on federal e-government Web sites: a multi-method, user-centered evaluation of accessibility for persons with disabilities. Gov. Inf. Q. 23(2), 169–190 (2006)
W3Techs: Usage of content management systems. https://w3techs.com/technologies/overview/content_management. Accessed 11 Dec 2019
W3C: WAI-Tools. https://www.w3.org/WAI/about/projects/wai-tools/. Accessed 11 Dec 2019
WADcher: Web accessibility directive decision support environment. https://wadcher.eu. Accessed 11 Dec 2019
W3C: WAI-Guide. https://www.w3.org/WAI/about/projects/wai-guide/. Accessed 11 Dec 2019
Funka: We4Authors. https://www.funka.com/en/projekt/we4authors/what-is-we4authors. Accessed 11 Dec 2019
Nekkra, U. G.: Kraken.io. https://kraken.io. Accessed 12 Dec 2018
Mediafox Marketing s.r.o.: Optimizilla. https://imagecompressor.com. Accessed 12 Dec 2018
WordPress Foundation: WordPress. https://wordpress.org. Accessed 12 Dec 2018
Open Source Matters Inc.: Joomla! https://www.joomla.org. Accessed 12 Dec 2018
Magento Inc.: Magento. https://magento.com. Accessed 12 Dec 2018
Drupal community: Drupal. https://www.drupal.org. Accessed 12 Dec 2018
North Caroline State University: All links must contain either text or an image with alt text. https://accessibility.oit.ncsu.edu/it-accessibility-at-nc-state/developers/accessibility-handbook/mouse-and-keyboard-events/links/all-links-must-contain-either-text-or-an-image-with-alt-text/. Accessed 10 Dec 2019
W3C: Web accessibility tutorials: labeling controls. https://www.w3.org/WAI/tutorials/forms/labels/. Accessed 10 Dec 2019
Soueidan, S.: Accessible icon buttons. https://www.sarasoueidan.com/blog/accessible-icon-buttons/. Accessed 18 Dec 2019
T. Page: Text justification—issues and techniques. https://accessible-digital-documents.com/blog/justified-text/. Accessed 20 May 2019
Instructure Inc.: TinyMCE Accessibility Checker Plugin (tinymce-a11y-checker). https://github.com/instructure/tinymce-a11y-checker. Accessed 17 May 2019
Wikipedia: WYSIWYG. https://wikipedia.org/wiki/WYSIWYG. Accessed 21 May 2019
Alexandrou, M.: Real-time find and replace. https://wordpress.org/plugins/real-time-find-and-replace/. Accessed 11 Dec 2019
van Westen, P.: ReReplacer. https://www.regularlabs.com/extensions/rereplacer. Accessed 11 Dec 2019
Shrivastava, T.: 13 Apache web server security and hardening tips. https://www.tecmint.com/apache-security-tips/. Accessed 13 Dec 2018
CVE Details: Joomla: vulnerability statistics. https://www.cvedetails.com/vendor/3496/Joomla.html. Accessed 13 Dec 2018
CVE Details: WordPress: vulnerability statistics. https://www.cvedetails.com/vendor/2337/Wordpress.html. Accessed 13 Dec 2018
CVE Details: Drupal: vulnerability statistics. https://www.cvedetails.com/vendor/1367/Drupal.html. Accessed 13 Dec 2018
WPBeginner LLC.: How to easily create a Multilingual WordPress Site. https://www.wpbeginner.com/beginners-guide/how-to-easily-create-a-multilingual-wordpress-site/
Open Source Matters Inc.: Joomla! Documentation: setup a Multilingual Site/Installing New Language. https://docs.joomla.org/J3.x:Setup_a_Multilingual_Site/Installing_New_Language
Open access funding provided by University of Pannonia (PE). We acknowledge the financial support of Széchenyi 2020 under the EFOP-3.6.1-16-2016-00015.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Csontos, B., Heckl, I. Accessibility, usability, and security evaluation of Hungarian government websites. Univ Access Inf Soc 20, 139–156 (2021). https://doi.org/10.1007/s10209-020-00716-9
- Government websites