1 Introduction

eHealth, online health information and healthcare services, are increasingly important for people who need healthcare. The World Health Organization (WHO) estimates that about 15% of the world population experience disability and that the number will increase due to demographic changes [1]. Article 25 in the UN Convention on the Rights of Persons with Disabilities (CRPD) declares that healthcare should be distributed equally to everyone and without discrimination [2]. Hence, it is important that eHealth is designed to be accessible for all people, including people with impairment. Research show that elderly people have higher prevalence of impairment than younger [3].

The Web Accessibility Directive (WAD) [4] is part of the work for an inclusive European Union (EU), aiming for all people to be able to take a fully active part in the digital society. The WAD regulated that the national EU member states legislation come into force on 23 September 2020 for public sector healthcare providers’ websites and eHealth services. The WAD was announced already in 2016 to give public sector bodies time to adapt their websites and web services to the upcoming legislation, in Sweden called the DOS-law. The DOS-law was decided by the Swedish parliament in November 2018 and implemented as national legislation in January 2019 [5]. The WAD regulates that public sector bodies in the member states should meet specific technical accessibility standards on public websites and apps [6]. The technical accessibility standards are presented in the harmonised European standard (EN) 301 549 [7], which in turn refers to the Web Content Accessibility Guidelines (WCAG) 2.1 on level AA [8]. There is lack of common ground in definitions regarding accessibility and an overlap between terms, e.g. accessibility and usability [9, 10]. The WAD states that “accessibility should be understood as principles and techniques to be observed when designing, constructing, maintaining, and updating websites and mobile applications in order to make them more accessible to users, in particular persons with disabilities” [4]. WCAG 2.1 has been critiqued for not providing enough guidance on for example cognitive accessibility [11, 12]. In this paper, we consider requirements in the WCAG 2.1 as accessibility requirements, acknowledging that there are more accessibility requirements than stated in the WCAG 2.1. We do not investigate the accessibility requirements that are not mandated by the regulation, i.e. requirements on level AAA.

The WAD also requires public sector bodies in all member states to provide an accessibility statement for each website and app, and a feedback mechanism so that users can report accessibility issues or request inaccessible content in an accessible format [13, 14]. The WAD is very specific about the content of an accessibility statement [14] and that it should be easy to find on the website or in information about an app. The accessibility statement is supposed to inform users with impairments whether the website or app complies with the law and if not, what kind of issues the user can expect, as well as a dated plan for when improvements will take place. As part of the accessibility statement, the provider of a website can claim ‘disproportionate burden’, arguing that the correction of deviations from the law would be too costly. The accessibility statement should inform the user on how the accessibility evaluation was conducted and when it was done. Further, the date of the publication of the statement is mandatory. The WAD also regulates that the member states should monitor accessibility in public sector websites and apps and report the results to the EU.

The Agency for Digital Government in Sweden (DIGG) is the public authority that acts as the enforcement procedure and monitors how the public sector bodies in Sweden comply with the DOS-law [15]. The DIGG also provides information and guidance on the matter [16].

Around the world, several researchers have evaluated eHealth websites conformance to WCAG using automated accessibility assessment tools. Research on accessibility issues in eHealth was also conducted prior to 2016. For example, in 2005, Mancini et al. pointed out accessibility issues in 76% of 170 Italian Local Health Authority websites [17]. Martins et al. [18] evaluated 697 Iberian eHealth websites showing that none was compliant to WCAG 2.0. Alhadreti [19] evaluated twenty Saudi Arabian hospital websites homepages with the automated accessibility assessment tool AChecker [20] showing that only 20% passed the WCAG 2.0 conformance test on AA level. Sarita et al. [21] evaluated six Indian healthcare websites using the automated accessibility assessment tools AChecker [20], WAVE [22] and TAW [23], showing that the evaluation results differed between the tools. They pointed out that no automated tool can assess all aspects of accessibility conformance to the WCAG. The need of combining automated tests and human feedback is confirmed by Sik-Lanyi and Orbán-Mihálykó [24] who evaluated 99 healthcare-related websites from nine European countries using two automated tools AChecker [20] and Nibbler [25] combined with human feedback through evaluation by experts guided by a questionnaire. Mason et al. [26] assessed the 139 globally top ranked health websites, according to the Alexa database, regarding accessibility using the automated accessibility assessment tool WAVE [22] to evaluate web accessibility. The evaluation revealed that 91.3% had detectable WCAG 2 failures. Alismail and Chipidza [27] investigated 54 US COVID-19 vaccine registration websites homepage conformance to WCAG 2.0 and WCAG 2.1 using AChecker [20] and WAVE [22]. A full website evaluation was performed with SortSite [28] on the 22 websites that passed the evaluation of their homepage according to pre-set criteria. The results show that five webpages passed the AChecker test with no detected issues and three websites had no detected issues by WAVE. However, the websites with no detected issues differed in the two evaluations. Only two websites fully conformed to WCAG 2.1 on all levels according to the evaluation conducted with SortSite [27]. Yu [29] evaluated a dedicated COVID-19 information website in Australia using WAVE [22] showing that only a few violations to WCAG were corrected on the website between March 2020 and October 2020. An evaluation targeting 58 university hospital websites in Turkey by Macakoğlu and Peker [30] reports low compliance to WCAG 2.1 evaluated with TAW [23], but also a high number of websites with at least one dead link tested with the Dead Link Checker tool [31] and that 36% of the websites failed the Google Mobile-Friendly Test [32]. Previous research has also evaluated accessibility for specific target groups. For example, Arief et al. [33] evaluated 38 websites with information on dementia by using the automated accessibility assessment tools, AChecker [20] and AXE [34]. Worth noticing is that Arief et al. [33]and Mason et al. [26] assessed readability and Mason et al. [26] considered this as being part of accessibility, but WCAG 2.1 and previous versions do not provide guidance on how to evaluate readability.

The EU directive that regulates public authorities to evaluate accessibility on their websites and publish an accessibility statement presents an opportunity for users to get information about accessibility on any of the websites targeted by the directive. There is only a small body of research on accessibility statements. Lewthwaite and James [35] argue that since the accessibility statement must describe accessibility barriers in non-technical terms, how to contact for assistance and how to file complaints with the public sector body or with the government’s enforcement procedure, it should be easy for people to notify deficiencies in accessibility. They also suggest that accessibility statements contribute to a raised awareness of accessibility for website users and public sector bodies, and that accessibility statements could empower people, experiencing exposure to inaccessible content, to be more active in filing complaints. Access to the digital society is discussed as a human right in the CRPD [2] and by several scholars [36,37,38,39]. Lewthwaite and James [35] describe the process regarding accessibility statements as a step to ensure compliance with the CRPD.

The public healthcare system in Sweden is available for all people and organised by the Swedish healthcare regions, who have their own self-governing regional authorities. Hence, the healthcare regions’ websites are an important source for health information to the public. Inera is the provider of general healthcare information in Sweden and also a major part of the healthcare regions eHealth services, hosted on an online national portal called 1177.se [40]. The portal consists of an open section with healthcare information and eHealth services accessed with login by electronic personal identification.

The objective of this study was to evaluate how healthcare providers in Sweden have applied accessibility statements on their websites as regulated by law.

2 Method

2.1 Study design

This study had a descriptive study design. A mixed methods approach was applied for data collection and analysis, by quantitative descriptive data analysis of the healthcare providers’ accessibility statements compliance to requirements and qualitative data analysis of the written information provided in the accessibility statement. Quantitative and qualitative data were analysed separately. The results of the qualitative and quantitative data analyses were then merged and integrated in the interpretation of results.

2.2 Sample

A purposeful sample of healthcare providers was chosen to cover all major online healthcare information and eHealth services available to Swedish citizens: i.e. all websites of the Swedish healthcare Regions; the eHealth services on websites in the national portal 1177.se provided by Inera; and large private healthcare providers and public hospitals that have their own websites (Appendix B).

2.3 Study procedures

The first author collected three versions of the accessibility statements from the included healthcare providers of online healthcare information and eHealth services: (1) the most recent version available on the website, (2) the oldest version found by Wayback Machine [41]; and (3) an intermediate version found by Wayback Machine. The accessibility statements were downloaded in pdf-format. The data collection was conducted in 2022 between 14 March and 12 December. The search was conducted in Swedish.

To find the accessibility statement, the first author screened the healthcare providers’ websites in a stepwise procedure. Steps 1 and 2 were conducted for all websites. Steps 3 to 5 were conducted in the presented order, and the search ended on the step where the accessibility statement was found:

  1. 1.

    Search for the accessibility statement in the internal search function.

  2. 2.

    Altering search terms to terms related to accessibility.

  3. 3.

    Manual inspection of page header, page footer, link to ‘about the website’ or similar sections.

  4. 4.

    Google search combining healthcare provider's name and the term ‘accessibility statement’.

  5. 5.

    Requesting accessibility statement by email from the provider of the website.

The oldest accessibility statements were found by using the Wayback Machine, a historical digital library of internet sites, provided by the non-profit organisation The Internet Archive [41]. The Wayback Machine makes it possible to visit archived versions of publicly available web pages. The URL for every accessibility statement was used as a search term in Wayback Machine. The oldest version of an accessibility statement for each healthcare provider was downloaded together with one version from the intermediate period of May to August in 2021, if available.

An assessment protocol for collecting data on requirements for accessibility statements was developed based on review of the WAD [14], the DOS-law [5] and instructions issued by the DIGG [42], as well as the reporting templates provided in the WAD and by the DIGG (Appendix A). The extraction of requirements was an iterative process performed by the first and last author, since the instructions from the WAD, DOS-law and the DIGG are slightly differently presented and in different wording. The first and the last author returned to the material in loops to develop and refine the assessment protocol by comparing texts from the three sources. The second and third author acted as quality assessors and participated in elaborating the assessment protocol. The iterative process for creating the assessment protocol is presented in Fig. 1.

Fig. 1
figure 1

Flow chart of the iterative process for creating the assessment protocol by reviewing sources for accessibility statement requirements

Full size image

2.4 Data collection

The first author used the assessment protocol for collecting quantitative and qualitative data from the accessibility statements regarding how public sector healthcare bodies in Sweden adhered to the DOS-law from the date the legislation came into force until the current date.

The data were collected in three sections:

  • Section 1, checkpoints 1–2: presence of accessibility statements at different time points by assessing if there was an accessibility statement published before or on the date 23 September 2020 when it was first required by the law. Assessment of versions of the accessibility statement from three time points between 2020 and 2022, to assess if there were changes made over time in the accessibility statement and to identify reported improvements in accessibility;

  • Section 2, checkpoints 3–13: quantitative assessment of the level of compliance to the requirements for accessibility statements;

  • Section 3, checkpoints 14–16: qualitative assessment of the comprehensiveness, understandability, and usefulness of the description of accessibility issues in the accessibility statement.

2.5 Data analysis

The quantitative data collected in Sects. 1 and 2 of the assessment protocol were analysed with descriptive statistics and presented by numbers and proportions.

The qualitative data collected in Sect. 3 of the assessment protocol were analysed for the comprehensiveness, understandability, and usefulness by a thematic analysis [43]. Since WAD states that the accessibility statement should be detailed, comprehensive and clear but do not provide any definitions of those concepts, the research group developed a set of basic requirements that could be applicable for accessibility statement (Table 1). The research group used prior experiences from the fields of accessibility and human–computer interaction in how to produce accessible and understandable information when developing the basic requirements. Data were collected directly from the included websites and by the Wayback Machine. Familiarising with the data started during data collection from the websites and in the Wayback Machine. The collected data were then read several times and revisited during the iterative thematic analysis process. A Miro [44] board was used to arrange data thematically and for all authors to participate in the analysis. Screenshots of the accessibility statements were presented on the Miro board. The first and last author performed the preliminary analysis. Then, all authors participated in discussing and elaborating the analysis until consensus was obtained.

Table 1 Qualitative assessment of the comprehensiveness, understandability, and usefulness of the description of how accessibility issues were presented in the accessibility statement
Full size table

3 Results

3.1 Quantitative assessment of the level of compliance

In total, 37 healthcare providers’ websites and eHealth services were assessed for accessibility statements: all 21 Swedish healthcare Regions, eight eHealth services in the national portal 1177.se provided by Inera, five large private healthcare providers and three large public hospitals that have their own websites. A published accessibility statement was found on 36 healthcare providers’ websites. The healthcare provider that did not have any accessibility statement was excluded from the analysis. The result from the remaining 36 healthcare providers’ websites is summarised in Table 2.

Table 2 Results of the quantitative analysis of the healthcare providers’ websites that had an accessibility statement, n = 36
Full size table

The 36 websites that had an accessibility statement, reported accessibility issues both in the oldest and most recent published accessibility statement. No website claimed to fully comply with the DOS-law in any version of the accessibility statement. Partial compliance was claimed by eleven healthcare providers and no compliance by six. Nineteen healthcare providers did not report the level of compliance or failed to follow the instructed wording. These are marked as ‘other’ in Table 2. Upon request, the provider with no published accessibility statement responded that they were aware of the legislation but did not have any accessibility statement. All accessibility statements explained what parts had accessibility issues.

Twenty-four websites provided a date for when the latest conformance evaluation of the website was conducted. Nineteen websites provided a date for when the accessibility statement was updated. In 15 accessibility statements, the first published version was also the latest.

In six accessibility statements, more issues were reported in the most recent published statement than the first published version. In nine accessibility statements, fewer issues were reported in the latest accessibility statement. In four accessibility statements, the number of issues was the same, but some issues had been resolved and others added. A description on how the conformance evaluation had been conducted was provided in 28 accessibility statements. The most common means of evaluation was internal evaluation (n = 12), followed by a combination of external and internal evaluation (n = 9), and external evaluation (n = 7). External evaluation indicates that accessibility experts have been brought in to do the evaluation. Disproportionate burden was claimed in eleven accessibility statements. The accessibility issues listed as disproportionate burden were pdf-documents, videos, forms, defining language element in translations to other languages, maps, and older systems. Content outside the scope of the law was claimed in three accessibility statements. The listed categories of content outside the scope of the law were older videos and documents.

A plan for when inaccessible issues should be resolved was missing in 22 accessibility statements. Four healthcare providers reported that they planned to resolve accessibility issues during 2021 with no explanation for the delays. Ten healthcare providers reported that their plan was to solve the remaining issues during 2022, and one healthcare provider reported that they planned to solve the issues in 2024. Among these fourteen, two healthcare providers reported different dates for when they planned to resolve specific issues. Of the 36 accessibility statements, 19 were found through a link in the page footer and 26 through an internal search by using the term ‘accessibility statement’. Seventeen statements could be found both by a link and a search. Alternative search terms were needed to find the accessibility statement on seven websites. In one website we could not find a search term that provided a result despite a link to the accessibility statement in the page footer. A manual review of the website was needed in two websites which neither provided an accessibility statement by internal search nor by a link from the home page. In those cases, one statement was found through a link called ‘Accessibility’ and the other through a link called ‘About the website’.

A feedback mechanism to the healthcare provider by a web form was presented in 20 accessibility statements. A feedback mechanism as the possibility to send an e-mail was presented in 17 accessibility statements, and as the possibility to use a phone number was presented in nine accessibility statements. Ten accessibility statements presented more than one feedback mechanism. Information on and link to the enforcement procedure provided by the DIGG was presented in 32 accessibility statements.

3.2 Qualitative assessment of the comprehensiveness, understandability, and usefulness

The overall usefulness was constructed by an analysis of comprehensiveness, clarity, and user orientation. The qualitative assessment of the information provided in the accessibility statement (checkpoints 14–16) resulted in three accessibility statements that were regarded as very useful for people with impairments. That is, people with impairments could probably use the information without explicit knowledge about html-code, WCAG or other technical or juridical wording, relate the information to their specific impairment and/or use of assistive technologies, and understand the potential personal issues when trying to access the healthcare provider’s website.

3.2.1 Comprehensiveness

Regarding comprehensiveness, less than half of the accessibility statements (n = 15) provided an explanation of what kind of accessibility issues the user could expect and how frequently they appeared. The accessibility statements were interpreted as having low comprehensiveness when presenting explanations such as ‘Specify in code the content for each part of the page’ with reference to WCAG 2.1, criterion 1.3.1, or ‘There are some css-files that still have warnings of validation errors’, or ‘On this site pdf-files could occur that are not correctly constructed’, without explaining what kind of errors the user could expect. An example of an accessibility statement interpreted as a comprehensive statement declared ‘Some pdf-files do not have the correct structure to be read by text to speech technology, they might present an incorrect reading order and tab-order. Some pictures might lack alternative text descriptions.’ The latter statement would help users with both reading impairments and vision impairments to decide whether to read the documents or contact the healthcare provider to request more accessible content.

3.2.2 Clarity

Regarding clarity, only a few accessibility statements presented accessibility issues in plain language with explanations of unusual or less common words. Clarity was also obstructed by a lack of structure with no headings or hard to understand headings, and not using bullet lists when presenting multiple issues. Clarity was also often obstructed by using technical words only understandable if the reader is familiar with the WCAG standard and with wording related to HTML-code. Accessibility issues were often not presented in a way that allowed major accessibility errors to be easily found, for example presented at the beginning of the text. Instead, minor and major accessibility issues were presented in a random order.

3.2.3 Usefulness

Half of the accessibility statements presented accessibility issues with a clear user orientation. These accessibility statements used for example headers separating issues related to specific impairments such as vision, hearing, and reading. The accessibility statements that we interpreted as failing in user orientation used for example the overall WCAG principles perceivable; operable; understandable; and robust, as headings when presenting accessibility issues. The reason to interpret use of the WCAG principles as failing in user orientation was that said principles are probably not known by most users.

The accessibility statements provided information about inaccessible content presented from different perspectives. These perspectives were presented in headlines, to guide the user to understand where the issues could appear. One approach was to describe accessibility issues related to impairments (n = 7) or mixing information relating to impairments and content (n = 4). The impairments described in the accessibility statements were vision loss (n = 7); partial visual loss (n = 9); hearing loss (n = 5), partial hearing loss (n = 5); cognitive impairment (n = 4); fine motor function loss (n = 5); reduced muscle strength (n = 5); reduced colour vision (n = 3); and reduced mobility (n = 3). A second approach was to relate accessibility issues to specific assistive technologies (n = 7), such as screen readers (n = 6) or the use of keyboard navigation (n = 4). A third approach was to inform about inaccessible content without a headline or with an unspecific headline (n = 11). A fourth approach was to relate accessibility issues directly to a specific type of content, for example videos (n = 5). One accessibility statement presented accessibility issues with headlines referring to the main principles in the WCAG, i.e. perceivable; operable; understandable; and robust.

4 Discussion

Our most important findings of this study are that no fully accessible websites were found, even though the regulation has been in place since 2018, only little progress regarding accessibility is shown in the accessibility statements from 2018 until 2022, and that all of the evaluated websites fail to report and design the accessibility statements according to instructions in the model accessibility statement.

The websites within the scope of this study were supposed to comply with the law by the 23 September 2020. Not one of the 37 healthcare providers complied with the law when it came into force, and nor did any at the time of data collection for this study. This is in line with research published by Alajarmeh et al. [45] who showed that many critical accessibility barriers remained in public healthcare websites in April 2020 in the 25 countries listed as most affected by COVID-19 at the time. This is also confirmed by Acosta-Vargas et al. [46] who found frequent accessibility errors on the seven most visited healthcare websites in the world in 2021.

The findings of our study add to the previous knowledge, by showing that alongside non-compliance to the accessibility requirements stated by law, there are also shortcomings related to the publishing of accessibility statements. Our results are in line with Lynn et al. [47], who showed that most of the Irish Local Government websites had published an accessibility statement, but the form and details varied. Our study shows that no healthcare provider met all the requirements for the accessibility statement as stated by the law. Two accessibility statements met the quantitative assessment checkpoints 4–13 but failed the qualitative assessment checkpoints 14–16. Overall, only three of the 36 included accessibility statements met the qualitative criteria in our assessment protocol.

To the best of our knowledge, this is the first suggested assessment protocol on how to systematically evaluate how accessibility statements comply with the WAD. Accessibility statements alongside the feedback mechanisms are an important pillar for the whole construction of the WAD. Our findings indicate that neither the instructions for the accessibility statements nor the intentions of the WAD are being followed. It seems that the accessibility issues reported in 2020 remain in 2022, indicating that there is no or only minor improvements towards conforming to the law.

Our findings also show that there is an inconsistency regarding the terminology on how healthcare providers reported compliance with the WAD. The instructions from the EU and DIGG state that compliance should be declared using the labels full, partial, or not compliant. However, we found that healthcare providers used other ways of expressing the lack of compliance, i.e. ‘mainly full’ or ‘we did not manage to meet all the criteria in the WCAG’. We argue that using the correct term will make it easier to evaluate the different websites and understand the level of compliance. There is a risk that the level of compliance is not stated correctly in many accessibility statements. Ten healthcare providers claimed to be partially compliant but listed several accessibility issues. According to the instructions from the DIGG [42], partial compliance should be claimed when most of the pages on the websites are fully accessible or when only some inaccessible content is present. In our understanding of the instructions, the appropriate level of compliance should have been ‘Not compliant’ for all ten healthcare providers. Without further instructions from the DIGG there may be room for different interpretations on when to declare the level as partially compliant.

Most of the accessibility statements did not present a plan for when remaining accessibility issues are to be resolved. Seven providers stated already-passed dates for resolving accessibility problems. We found some good examples of work conducted to solve accessibility issues, with clear progress shown, but those were exceptions. There were also good examples of healthcare providers adding information in the accessibility statement that is not mandatory, such as informing the public that there is ongoing education of staff in how to create accessible documents.

Several providers claimed a disproportionate burden to explain why they had not adapted some of their inaccessible content. It is important to notice that the regions and the private healthcare providers operating on public funding are some of the most resourceful organisations in Sweden, spending about 11% of the Swedish gross domestic product [48]. We argue that none of the healthcare providers should claim disproportionate burden.

The construction of the WAD provides citizens with an opportunity to complain about inaccessible content and demand help. Every member state must implement an enforcement procedure. The EU mandates all member states to monitor the development and to report status and progress. It is too early to tell how the monitoring part will play out. In a seminar (hosted by Funka in Stockholm, 8 April 2022), the general director of the DIGG presented that they will apply a soft approach to public sector bodies not complying with the law [49]. In the first published national monitoring report for the period 2019–2020, the DIGG reports that out of 307 controlled websites no one fully complied with the law, 94 partially complied and 207 did not comply with the law [15]. According to the same report, 43 complaints had been filed with the DIGG between September 2020 and November 2021 and out of those, two had resulted in supervisory decisions from the DIGG. The WAD regulates that the DIGG can issue fines to public sector bodies not complying with the law, but in a response to our question, DIGG confirms that no such fines have yet been issued (personal communication with the DIGG in December 2022). It is interesting to compare this soft WAD approach to the rigid application of the General Data Protection Regulation, GDPR [50]. The GDPR gives no opportunity to declare disproportionate burden and there are high fines for those who do not comply [51].

Even when a feedback mechanism and the enforcement procedures are technically in place, it does not automatically result in the WAD becoming a tool and a channel for disabled people to demand accessible information and online services. The potential of a WAD presented as an emancipatory tool, as argued by Lewthwaite and James [35], is not yet realised. This is also confirmed by the EU in their review of the application of the WAD, reporting that few citizens file complaints [52]. To raise the awareness of the possibility to complain, we propose that monitoring authorities should launch information campaigns to the public and that certain stakeholders, such as the disability movement, need to alert their members to this opportunity.

The low number of accessibility statements that were deemed useful for a user with an impairment, may indicate that healthcare providers have not fully acknowledged the intention behind the accessibility statement, i.e. that the statement should be useful for a user with impairment. The content was often difficult to understand, for example when containing technical terms. Another problem is that some claims were very broad, for example the fact that several healthcare providers declared the presence of inaccessible pdf-documents on their websites. That is an unspecific statement and when screening a sample of such documents, we saw that the problems were often related to interoperability issues for screen readers. Simply declaring that a document is inaccessible without explaining the issues in more detail, creates a risk that some users refrain from trying to use the document, believing they cannot use it at all. The healthcare providers apply different approaches to presenting their accessibility issues, i.e. listing barriers from the perspective of impairment or assistive technology.

The overall result of the thematic analysis is that although there are detailed instructions on how to describe accessibility issues, and templates to use, the results for the 36 healthcare providers are diverse. In most cases, the accessibility statements cannot be regarded as useful for people with disabilities. Only three were good enough according to our analysis and even in those accessibility statements there was potential for improvements. We suggest that there is a need for a combination of approaches to create comprehensive, clear, and user-oriented statements and that no one has really figured out how to do that so far. An important next step in developing useful accessibility statements is to refine the descriptions of the accessibility issues and the templates for accessibility statements, by working closely together with the users who need the information.

The WAD instructs public sector bodies to use understandable text to explain inaccessible content. We suggest that adding pictures or symbols to plain language text is beneficial for understanding. Pictures and symbols are useful additions to text [53,54,55,56] and video presentations of the accessibility issues would also improve understandability.

A question outside of the scope for this study did arise during the research: can the declarations presented in the accessibility statements be trusted? As an example, one region reported only one accessibility issue in their accessibility statement, stating that the navigation menu was inaccessible for users with small screens. Since 90% of the Swedish population use the internet regularly through a smartphone [57], such an accessibility issue can have major implications for the users, so it is justified to highlight it in the statement, even though it could be regarded as a general usability problem. However, we could easily find several non-declared accessibility errors by doing a quick manual accessibility inspection. This puts the trustworthiness of the accessibility statements in focus. What needs to be further analysed is whether the accuracy of the statements can be trusted, that is if what is being declared as a deviation from the law also represents the true accessibility status of the website or app. Few accessibility statements provided detailed information on the evaluation of accessibility. Only a few declared what automated tools they had used, and whether they had only been using automated tools or also combined the use of tools with manual inspections of the code. In a few cases, the test protocol was posted on the website. W3C has clear recommendations on how evaluations should be conducted to provide a full conformance WCAG evaluation [58]. Since different accessibility assessment tools may not always detect the same accessibility issues [21], we argue that there needs to be information about the combination of tools used, and how the manual inspection has been carried out in the accessibility statement. We believe that would increase the transparency and the trustworthiness of the accessibility statement.

Since we found some differences between the WAD and the Swedish DOS legislation and the instructions from the DIGG, it is possible that there could be differences also in comparison with other EU member states’ national legislation and the WAD. We suggest that our assessment protocol is feasible for future assessments of accessibility statements, but it might need adaptations if there are differences in how the WAD has been implemented in each national legislation.

The requirement on the possibility to locate the accessibility statement by using internal or external search engines is derived by us from the WAD requirement that an accessibility statement should be easy to find. The WAD does not define ‘easy to find’. To be more precise on ‘easy to find’, we argue that since users adopt different strategies to find information, it should be possible to both navigate to the accessibility statement by following links, or to find it through a search engine. In several cases, it was hard to find the accessibility statement since there was no direct link from the main page header or footer. Seven healthcare providers did not use the term at all. Instead, they had replaced it with other wordings. Sometimes the chosen expression was shorter (i.e. accessibility instead of accessibility statement).

The instructions on how to present accessibility issues for visitors on websites differ in the instructions provided by the WAD and by the DIGG. The WAD has an optional section where they recommend describing in non-technical terms how the content is inaccessible [14]. The example provided by the WAD is: ‘The login form of the document sharing application is not fully usable by keyboard (requirement number XXX (if applicable))’ [14] (Sect. 2).

The DIGG has instead a mandatory section where the instruction is to describe the inaccessible content as a problem in a user situation, and to give information as to which impairment the inaccessible content is related to.

4.1 Strengths and limitations

A strength of this study is that we used a structured protocol to assess the accessibility statements. The protocol contributes to reproducibility, trustworthiness, and validity of the study. Another strength is that our sample includes websites from all healthcare regions in Sweden and from the national eHealth platform. The results therefore reflect the state of healthcare information and eHealth services provided for people living in all parts of Sweden. A potential limitation of the study is that the Wayback Machine includes webpages from dates when it was crawling the web page and it might not have caught every time a page was updated, creating a risk that we have found an older but not the oldest version of the accessibility statement.

Since the law regulates to comply at the level AA, some accessibility requirements are left out from the regulation, for example sign language that is on level AAA. Our study aimed to evaluate how the accessibility statement was implemented and therefore, we did not investigate accessibility requirements outside the scope of our study.

Since there is no definition in the WAD on how to evaluate if an accessibility statement is “comprehensive, detailed and clear” [4], we suggest that our approach to use Comprehensiveness, Clarity, and User orientation as our basic definitions used in the qualitative evaluation should be further elaborated in future research. Furthermore, users that have impairments should be involved to check and validate the qualitative assessment of the accessibility statements.

5 Conclusion

Providing accessible digital public services to all citizens has, since 2020, been mandated by an EU legislation. We have analysed a purposeful sample of Swedish healthcare providers’ homepages and national eHealth services, to find out how well Sweden’s healthcare providers meet the intentions of the legislation. All but one of the 37 evaluated healthcare providers published an accessibility statement. None of them fully met the requirements for the accessibility statement, and not one complied with the intention of the law, that is to provide accessible health information and eHealth services to all citizens.

The possibility to declare no or partial compliance to the law, the possibility to claim disproportionate burden and the absence, so far, of strong enforcement procedures to date, creates a standstill situation. There is a risk of healthcare providers performing tokenistic, symbolic actions, e.g. publishing an accessibility statement with no real intention to abide by the law. To a healthcare perspective this is counterproductive, since people with impairments are already an underprivileged group in need of frequent interaction with the healthcare system. Pushing this group away from eHealth towards other means for the provision of healthcare could be far more costly than solving the digital accessibility issues.

Future research, in close collaboration with users with impairment, should be conducted to improve the instructions for the accessibility statements in order that they be useful for people with impairments. Future research should also investigate the accuracy of the accessibility statements.

The assessment protocol developed for this study is suggested as feasible for use in future assessment of accessibility statements.