For this study, 18 semi-structured interviews were conducted with stakeholders. In the coding process, the answers of the stakeholders were linked back to research model. For every occurrence of a factor during the interview, the coders ticked the assigned code.
Table 3 is an overview of the factors coded from the interviews with the stakeholders. The perceived level of influence is ordered as (1) no influence, (2) medium influence and (3) high influence.
The category “high influence” indicates factors that are mentioned by multiple stakeholders and where stakeholders indicated that this was an important factor influencing the final result.
In the following paragraphs, the results are discussed per category.
Which adoption factors are influencing the implementation of accessibility guidelines according to the stakeholders? Table 4 lists the number of quotes coded as adoption factors.
Not all factors from the models came forward in the interviews with the stakeholders. This is interesting because based on the literature and existing models one would expect these factors to have a significant impact.
Adoptions factors that were not named as influential by the stakeholders were:
The extent to which the accessibility standards are perceived to be easy or difficult to adopt and implement is covered in the factor “perceived complexity.” It is an indication of how the stakeholders perceive the accessibility standards. Stakeholders indicated that they experienced the accessibility standards as complex. Six of them referred to the amount of work necessary to implement the standards, and 10 indicated they would expect to have problems applying the standards to video on the web. Three out of the six municipalities have the perception that the standards concerning video are too strict. Some named the complexity of applying them to interactive maps, text and/or images.
Municipalities that have successfully implemented the standards perceive the standards as less complex than municipalities that do not yet meet the standards requirements. Also they are more positive about the standards. It is not always clear whether the perception of complexity is based on real experiences and a detailed study or knowledge of the standards, or an expression of hearsay.
This factor describes the availability of information about the technical possibilities of accessibility including clear guidelines, explanation and toolkits that can facilitate the adoption and implementation. Some of the stakeholders looked the subject up on the web, and others followed professional training courses or asked experts for information. Information was also provided by the providers of content management systems (CMS) and IT systems and infrastructure for municipalities. Some CMS systems include a check for accessibility that prompts editors before the page is published. For example, if an editor forgets to add a description to an image, that image will be difficult to understand for blind people. The system will then prompt them and ask for input. Editors from 5 municipalities indicated this ‘warning system’ as a factor that influenced their implementation of the standards. This, however, does not automatically mean that the website will be accessible. Even systems that have this ‘warning system’ in place and where editors indicate this as a factor still have inaccessible pages with these same errors. Both editors and suppliers indicate that the problem causing this is not with them. Availability of resources makes it easier to adopt and implement the accessibility standards, though availability does not automatically lead to adoption and implementation. Stakeholders need to see the importance and their responsibility before they look for these resources.
Stakeholders indicate that compatibility influences the decision to develop a new website and/or to apply accessibility standards. Compatibility describes the ease to adopt and implement the accessibility standards with existing infrastructure or the existing web services. It is interesting to conclude that the factor compatibility is only mentioned by stakeholders from municipalities that have successfully implemented the standards. The municipalities that mentioned this factor have not only implemented the standard but also changed their web service orientation more toward the citizens. This change of orientation could be a reason for the increase in the perceived importance of accessibility standards. Sometimes the change of orientation led to the need for a new infrastructure, making the choice to adopt and implement accessibility standards easier.
This describes the involvement of external agents with a degree of power on the internal adoption process. These agents include governments, programs, persons or other mandates. Five municipalities mentioned this factor. They named agents such as inspection/evaluation organizations, an authority like KING (Quality institute of Dutch municipalities), especially the iNUP program (National Implementation Program) as influencing the process. The iNUP program is one of the main sponsors of adoption and implementation named by all except one municipality. As this is a particular sort of legislation, it will be further explained when discussing external factors.
Web design factors
The interviews with the stakeholders were analyzed for all references to web design factors. Table 5 lists the number of quotes coded as web design factors.
This describes what a municipal stakeholder (involved in the design process) knows about the accessibility standards. Within the design process, this factor received the highest number of quotations by stakeholders (79). Managers do not always have prior knowledge of accessibility before they start the process. The project leader of the largest municipality solved this by hiring an external accessibility expert and including accessibility standards clearly in the contract as a requirement and a responsibility of the suppliers.
The developers showed very different levels of knowledge. Some had an in-house expert, and some had built a website with a successful implementation of the accessibility standards before. The websites of three municipalities that scored a high level of accessibility were all developed by developers that (1) had successfully implemented the accessibility standards before and (2) had an internal accessibility expert.
Not all managers, developers and editors followed accessibility courses. Some managers hired an expert; some developers and editors asked their colleagues and read the guidelines themselves.
Stakeholders involved in the adoption and implementation of municipalities that have a low score on accessibility seem to have less knowledge of the accessibility standards.
Budget and costs
This factor addresses the planned and the actual costs incurred by a municipality on adopting and implementing accessibility standards. It is used both as a reason and as an excuse to adopt and implement the standards. Stakeholders in municipalities with a low score for accessibility indicated that the priority was lowered as a result of the available budget. They stated that when the project reached the end, it ran out of money, and accessibility standards were removed to a lower place on the to-do list. Stakeholders in municipalities with a high score for accessibility all had a special budget allocated to accessibility. One municipality with a high score for accessibility had to implement the standards after the website was developed and launched. The stakeholder describes that it took a lot of effort and extra costs to realize this. The interviews show that a higher allocated budget can facilitate the implementation. However, perceived benefits by management are a precondition. If management does not perceive sufficient benefits, priorities seem to move away from accessibility standards.
Internal and external benefits
The interviews show a list of comments about perceived benefits. Stakeholders describe what they perceive to be the benefits of adoption and implementation of accessibility standards. It is interesting that municipalities that score low on accessibility also seem to find accessibility standards less important. The webmaster of one website indicated that they have multiple channels to serve people and that people who are deaf can also visit the municipality if the website is not accessible.
Some comments about benefits include:
The website is also accessible on mobile devices
Show citizens that the municipality takes responsibility if necessary
If all people are able to use the website, they do not have to call or visit in person. That may save costs (‘all’ includes people with disabilities)
Citizens are more satisfied with the municipality (as it becomes easier to do transactions)
The guidelines can be used as a weapon to convince other departments or convince the management to develop a new website
The management of the website gets less expensive when the code is of good quality
Having a faster website
Having a better ranking in search engines
Stakeholders mentioned accessibility standards as an instrument to convince other departments and management to develop a new website. The reason they used this argument is not always primarily to adopt and implement the accessibility standards: One of the municipalities used this argument to develop a new website and then during implementation adjusted the development in such a way that it did not result in successful implementation of the standards. Nevertheless, stakeholders indicate the importance of (perceived) benefits.
Stakeholders support the literature findings when it comes to quality assurance. Quality assurance helps monitor the degree of adoption and implementation of accessibility standards. They do require a certain level of knowledge and motivation with the people involved. Some municipalities in the study hired experts to support them in this process. Quality assurance helps municipalities in many ways including the possibility to save costs on (extra) external inspections. By continuously monitoring the activities of their suppliers, they can prevent implementation problems in an early stage of the process, thus saving expensive repairs at a later stage.
The interviews with the stakeholders were analyzed for all references to organizational structure. Table 6 presents the number of quotes coded as factors related to organizational structure.
Quality of procurement
This describes the degree to which the guidelines are mandated in the procurement requirements and checking the accessibility standards skills of contractors before contracting them. All stakeholders indicate that accessibility standards were part of the requirements in the request for tender that was sent out. This is remarkable because only two of the municipalities did finally implement the guidelines. Stakeholders indicated that in the talks with potential suppliers, they all claimed to be able to implement the accessibility standards as required. During the process, it seemed that this was not always true. Municipalities did not always check these claims by the suppliers. Also, many stakeholders believe that the technical part of the website is conformant with accessibility standards, while this is not always true. This belief is mostly based on the suppliers/developers claims which are not checked for validity. Only one municipality asked the supplier to repair the non-conformance after they had the website checked.
Importance and priorities
During the interviews, the stakeholders mentioned importance and priorities as key factors in the process of adoption and implementation of accessibility standards. Importance and priorities play a central role in almost all the factors, having the highest co-occurrence with other codes (98). Therefore this factor was added to the coding list during the coding process.
This factor refers to the extent to which the responsibilities of a stakeholder are clearly assigned in the process so the stakeholder and other stakeholders are aware of their responsibilities. The interviews show that if responsibilities are not clear, persons involved start pointing to suppliers or to others persons from inside or outside the municipality for accessibility. Although in all municipalities one or more stakeholders are responsible for the website and the implementation of accessibility standards, none of the stakeholders interviewed indicate they felt a particular responsibility for the overall implementation of the standards. For instance, some stakeholders pointed to the built-in features of CMS systems, although they were sometimes not aware that these features do not cover all standards. Suppliers indicate that they never received a question about expanding the features to cover the standards. They did not take that responsibility themselves. One municipality employed an external project leader with the task to reach a high conformance level for the new website. This project leader indicates that it took much effort to check and push suppliers.
Many internal municipality stakeholders indicate that they did not have to report to their management.
Managerial commitment and decisions
This describes the commitment of management to implement the guidelines and to take necessary steps. This factor was mentioned by stakeholders in all disciplines. In municipalities where managerial commitment with the website and online services was high, stakeholders perceived fewer problems with adoption and implementation.
Non-management stakeholders play an important role in the commitment of managers. Managers indicate that they gave higher priority to implementation of accessibility as a result of advice or pressure from web and communication colleagues in the municipality. In the largest municipality, the stakeholders indicated that the subject was high on the priority list of the new website process thanks to the continuous interest of a municipality council member. This continuous interest created a feeling of urgency that helped keep the subject high on the agenda during the process.
In municipalities with a lower score on accessibility, stakeholders mentioned that they had to discuss with the manager many times to keep the subject on the managers’ priority list. Some indicated that management was in the end not interested in the subject but used it nevertheless as a reason to develop a new website.
This factor refers to collaboration and exchange of information and best practices with other municipalities to adopt and implement the guidelines. Stakeholders indicate that this collaboration can help reduce the cost of implementing accessibility standards, thus also facilitating the decision to adopt and implement. The developers of the largest municipality in the study set up a partnership with dozens of other municipalities. The stakeholders of that municipality experienced collaboration as a factor that determined the successful implementation of accessibility standards. They shared the cost with the other municipalities.
However, collaboration is not always an indicator for successful implementation. One of the municipalities selected to use an open-source CMS and joined a network of municipalities using this CMS. In this network, they shared techniques and modules including accessibility functions. Nevertheless, the end result of their work did not include a good implementation of accessibility standards.
Different interests between stakeholders within a municipality involved in the adoption and implementation process. This factor seems to have a negative influence. Four municipalities mentioned pluralism. Examples include departments having very different wishes with regard to the amount of information, interaction or reading levels on the website, but also differences of opinion about tasks and restrictions of the website. Moreover, departments may have different views on the adoption and implementation of accessibility standards.
This factor refers to the dependency on other people for advice, approval, content delivery, information, etc., influencing the continuation of the adoption or implementation. Interdependencies seem to have a negative influence: One example includes having to wait for text documents provided by other departments that have to be made accessible first.
Closedness addresses the degree to which advice or comments from external stakeholders are accepted. This factor was not indicated by stakeholders to be very influential.
Interviews with the stakeholders were coded for personal factors as displayed in Table 7.
Stakeholder influence and involvement
Stakeholders indicate that they are committed and can influence the adoption and implementation process. Almost all stakeholders found that they could influence the process (at least within their own department or task). As found earlier with the factor responsibility, most stakeholders believed that their website was conformant with accessibility standards. When confronted with the fact that this was not always true, the stakeholders of the municipalities that score low on accessibility pointed out that this was not something they could have influenced. When asked for details, they pointed to other departments or stakeholders.
Opinion on guidelines
The stakeholders have very different opinions about guidelines. They all agree that the guidelines are necessary and ‘a good thing.’ Of the municipalities that find that some of the guidelines are too strict (5), most perceive the guidelines about accessible video requirements as a problem. Their solution was sometimes even to remove the videos from their website or transfer them to other websites such as YouTube. Particularly municipalities that score low on accessibility mention the strictness.
Pride and ambition
Municipalities that successfully implement the accessibility standards are proud of what they accomplished and mostly ambitious to do more. On the other hand, municipalities that have not successfully implemented the standards perceive the standards as complex, less important and too strict.
Disability in circle
Knowing how people with disabilities use the web helps to understand the importance of applying accessibility standards. Stakeholders in 3 municipalities had this experience and support this factor for that reason.
The number of quotes made by stakeholders regarding external factors was analyzed. Table 8 presents the number of quotes coded as external influences.
Legislation on accessibility
In the Netherlands, there are many laws and regulations that directly or indirectly require accessibility. Mostly, this is in the form of equal opportunities for all citizens. One of the most important is a high-level agreement between all government agencies in the Netherlands that they should have finished the implementation of accessibility standards by the first of January 2015. This is described in the i-NUP government program .
This factor is related to the “sponsorship” factor. Five municipalities mentioned legislation aspects when talking about sponsorship; specifically the i-NUP program (National Implementation Program) was referenced as influencing the process. At the time of conducting this study, three of the municipalities in the study had reached the required conformance level (level AA).
Stakeholders indicate that legislation influences the perceived importance and the priority in the process. Stakeholders in municipalities that already conform with the standards are very positive about legislation, while stakeholders within municipalities that score low on accessibility perceive this legislation as complex and unnecessary. Stakeholders remark that other rules and legislation may compete with accessibility legislation and lower the priority. This can have a negative influence on the adoption process. Also, they remark that there is not yet a sanction for that. The interviews do not prove that legislation is a decisive factor though it is important in helping to convince other stakeholders and some stakeholders indicate that it is a factor that helped in placing the subject on the list of priorities.
Other rules and demands
Stakeholders from three municipalities mentioned the importance of other rules and demands that are not directly accessibility related but compete for a high placement on the priority list. These include rules for online security, privacy and even formats and other regulations. The vast number of rules and demands that can also apply to their websites could make it hard for small municipalities to meet all demands. This perceived competition between legislation, rules and demands can cause accessibility to move down on the priority list.
One expert already indicated that complaints by citizens put the subject on the agenda so they started working on the subject. Stakeholders from three municipalities also mention this factor as influential. Two municipalities made changes following complaints by citizens. When citizens do not complain, municipalities may wrongly presume that this means they have correctly implemented accessibility standards and thus not carry out further work on implementation. This indicates that citizens can influence the adoption and implementation. The interviews show that if citizens would complain more, this would have a positive effect on the adoption and implementation of accessibility standards.