Factors explaining adoption and implementation processes for web accessibility standards within eGovernment systems and organizations

Abstract

Local government organizations such as municipalities often seem unable to fully adopt or implement web accessibility standards even if they are actively pursuing it. Based on existing adoption models, this study identifies factors in five categories that influence the adoption and implementation of accessibility standards for local government websites. Awareness of these factors is important for stakeholders adopting and implementing web accessibility standards. To further develop and understand these factors, this study has identified and interviewed experts in the field of (organizational) accessibility. This has led to an extension of the existing models. The extended model was then validated by interviews with key stakeholders. The outcome of this study places existing adoption models in a new context. The result is an adoption model that contributes better to explaining adoption and implementation processes within eGovernment systems and organizations. This adoption model aims to better help local governments in the identification of factors influencing the actual adoption and implementation of web accessibility standards in their situation. The model explains how factors in the different categories contribute to the adoption and implementation of web accessibility standards. The model may also be applicable to the adoption and implementation of other guidelines and (open) standards within local government.

Introduction

The web has become an essential and ubiquitous part of our daily life, and it is continuing to converge with media and technologies, including television, mobile telephony and a multitude of different kinds of mobile devices at home, at school and in the workplace. The web is part of our daily social, political and economic life.

Municipalities increasingly make use of the power of the web, both for their own benefit and for the benefit of their citizens. In the Netherlands, citizens in many municipalities can arrange meetings or order products using the web or special online applications. This saves the municipality and the citizen time and money and provides users with a 24/7 opportunity to interact with the government. However, this also requires the channel to be accessible to everyone ‘regardless of disability.’

Accessibility is a fundamental aspect of the modern information and knowledge society that is recognized by the UN Convention on the Rights of People with Disabilities (UNCRPD) [1]. This convention has been signed and ratified by most EU Member States and many other countries in the world. Earlier, the European Council adopted Resolution 7087/02 about ‘accessibility of public web sites and their content’ that calls for adoption of accessibility standards by all EU Member states [2]. In 2006, the ministers of 34 European countries unanimously signed the Riga Declaration that formulates concrete targets for e-inclusion and requires ‘that all public websites are accessible by 2010’ [3]. The European Parliament has prepared a ‘Directive of the European Parliament and of the Council on the accessibility of public sector bodies’ websites.’ This directive requires conformance with accessibility standards.

As a result of the Riga Declaration, in 2006, the Dutch Council of Ministers adopted the ‘Besluit Kwaliteit Rijksoverheidswebsites’ [4] that requires new and existing central government websites to comply with web accessibility standards by 2011. Local and regional government agencies in the Netherlands signed separate administrative agreements with the government in 2008 and 2011 [5, 6] that require them to implement accessibility standards by 2015.

While these commitments have contributed to more awareness among stakeholders, the actual adoption and implementation of accessibility standards is still behind the target set by the EU. The EC-funded subsequent MeAC studies of 2006, 2009 and 2013 [7] show that the overall conformance of websites with the standards may be improving, but is still far below the targeted level of accessibility. This is also visible when comparing yearly accessibility monitoring of government websites in the Netherlands [8]. It is clear that there is progress; however, the overall result is still below the set target [9, 10].

Reports from the responsible Dutch minister to the Parliament indicate that many efforts have been made to support municipalities and actively help them implement the standards [9]. Examples of these efforts include a 4-year program including ambassadors with disabilities to create awareness, information about the positive aspects of implementing accessibility standards, a national urgency program (i-NUP) that supports municipalities in the process, an accessibility testing tool, a benchmark website, co-creation of best practices and examples and more. Still, the minister concludes ‘that municipalities, provinces, water boards, non-departmental public bodies and central government agencies websites fail to conform with the required quality and accessibility standards.’ He expects ‘that this will also be the outcome of the next reports except for central government websites.’ The minister proposes an approach that is less focused on testing conformance at the end of the process and one that is more focused on the adoption and implementation process [10].

This brings up the question: Which factors within municipalities influence the process of adoption and implementation of accessibility standards for websites?

Based on existing models in the literature and additional input from experts, this study will list possible adoption and implementation factors in a research model and then rank them using semi-structured interviews with stakeholders from different sizes municipalities and with different degrees of adoption and implementation of web accessibility standards. To establish the status of the adoption and implementation of web accessibility standards, experts will examine websites of participating municipalities.

The study will identify key factors that influence the adoption and implementation of accessibility standards for local government websites and place them in a new adoption and implementation model. The study will also identify the involvement and influence of key stakeholders. This will help stakeholders recognize web accessibility-related adoption and implementation processes within their own eGovernment system and organization. It will also support easier identification of the factors that influence the success or failure in their specific situation. The model may also be applicable to the adoption and implementation of other guidelines and (open) standards within local government.

Section 2 describes the factors that influence adoption and implementation of accessibility standards for local government websites as they can be found in the literature. This section also describes the additional factors proposed by the experts in semi-structured interviews and ends with a research model that is used for validation with the stakeholders.

Section 3 describes the selection of the websites of the municipalities in the study, the way they were evaluated for accessibility and the semi-structured interviews with the stakeholders.

Section 4 describes the results of the validation of the research model by the stakeholders.

Section 5 proposes a new adoption and implementation model for the adoption and implementation of accessibility standards for local government websites. It separately describes the factors most important for adoption and the factors most important for implementation. It examines the impact of the size of the organization.

Adoption of accessibility standards

The Web Content Accessibility Guidelines (WCAG) [11] can be considered as the worldwide reference technical standard for web accessibility. The Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C) publishes them. The guidelines describe how to make web content accessible to persons with disabilities. WCAG addresses multimedia content, interactive components, and rich and mobile web applications. It is an ISO standard ISO/IEC 40500:2012 since 2012.

There are many studies about the use, implementation, validity and testing of accessibility standards for the web. Some studies focus on the usability and validity of the standards [12–15], some on the test quality [16], some on evaluations using the standards [17–19], and some are comparative [20, 21]. There are also many studies that measure the actual status of accessibility of websites in certain countries or areas at a certain time. This includes research into the accessibility of government websites in the UK, Netherlands, Saudi Arabia and all member states of the European Union [7–9, 22, 23].

Some countries have legislation that includes accessibility standards. The UN provides the Convention on the Rights of Persons with Disabilities [1] not yet ratified by all countries. The European Parliament is preparing an Accessibility Directive. It is clear from the studies though [7, 8] that even after a decade of availability of standards and even in countries where there is clear legislation, not all municipalities have adopted and implemented the web accessibility standards.

There is not much literature to clarify why, after all the years that the accessibility standards have been available, the majority of municipality websites have still not adopted and implemented them. There is a literature that describes factors that may be of influence on the adoption and implementation, though they are mostly focusing on information systems, innovation or information and communication technology (ICT) implementation. Lazar, Dudley-Sponaugle and Greenidge [24] looked into perceptions with regard to web accessibility; however, the study is limited to webmasters, while the group of stakeholders involved in the process is larger [25].

Adoption in the context of accessibility standards is all about the actual acceptance and the use of a product or technology by its intended users. It is a well-known term when it comes to innovation processes. Bouwman et al. [27] describe the adoption phase as ‘the phase of investigation, research, consideration and decision making in order to introduce a new innovation in the organization.’ In this study, the innovation would be the accessibility standards. As one of the first researchers to look into the adoption process of innovations, Rogers created the ‘diffusion of innovations’ [26] model. He describes several phases of the adoption process in the context of innovation and the factors that influence that process. Other researchers [27–29] took Rogers’ model and applied it to ICT systems. For example, Hovav [29] described factors in the context of the adoption of the Internet Protocol version 6 (IPv6) standard. Giannoumis [30] specifically looks into the importance of the WCAG guidelines for other ICT standards.

Factors influencing adoption

In the literature, a number of adoption and implementation models and theories are proposed. Traditionally, adoption studies focus on innovations in organizations [26]. Nevertheless, they have also been applied to development and introduction of new media, telematics innovations [28], ICT in organizations [27] and Internet standards as described by Hovav et al in 2004 and 2011 [29, 31].

One of the adoption models most referenced is the ‘diffusion of innovations’ model provided by Rogers [26] in which he describes the ‘units of adoption.’ Rogers’ model is on innovation. This study will examine the factors proposed by Rogers in relation to the adoption process of accessibility standards for websites. Rogers describes a number of factors that influence the adoption process. This study includes the following factors from his adoption model into the research model:

• Relative advantage The degree to which adoption and implementation leads to any kind of perceived advantage. For example, improvement of the website or organization as compared to not adopting and implementing it;

• Compatibility The ease to adopt and implement the accessibility standards with existing infrastructure and possibly existing web services;

• Complexity or simplicity The extent to which the accessibility standards are perceived to be easy or difficult to adopt and implement;

• Observability The degree to which the adoption and implementation of the guidelines is visible to others.

Hovav et al. [29, 31] created a model describing the adoption of Internet Protocol version 6 (IPv6) in South Korea. They concluded that the most influential factors seemed to be the government strategy to create user demand and to pressure companies to adopt IPv6 [29]. Other factors like sponsorship and financial factors seemed to have less influence. He then describes ‘environmental conduciveness’ and usefulness of the features as important factors. While the factors within usefulness are the same as Rogers’ [26], environmental conduciveness adds:

• Network externalities Pressure to adopt and implement guidelines because other municipalities also have;

• Current infrastructure and sunk costs of already existing infrastructure The age of the current infrastructure or the existing CMS system can make it more difficult or expensive to adopt or implement the guidelines. For instance, if a new website has just been implemented, it is perceived that large changes are required to implement the guidelines;

• Related technologies Existence and use of other technologies or open standards that make it easier to adopt and implement accessibility standards;

• Communication channels and information The extent to which information about the adoption and implementation is available to stakeholders;

• Sponsorship Involvement of external agents with a degree of power, like government or other mandates;

• Resources/technical possibilities Availability of resources on accessibility including clear guidelines, information and toolkits that can facilitate the adoption and implementation.

Factors related to the design process

Accessibility standards have to be applied during the design process of any website. During the design phase, there could be many factors influencing the implementation of accessibility standards. Abdelgawad et al. [32] have designed a diagram in which they describe their perception of the accessible design process as a cycle. The diagram shows how accessibility can be integrated throughout the whole design process and includes a number of new factors that may influence the application of accessibility standards during the design phase, such as budget, quality assurance and knowledge (specifically training, knowledge and experience of the workforce involved in the process). This study adds their perceived factors to the research model. In their opinion, implementation of accessibility standards would clearly benefit if more time would be spent on internal quality assurance. Kline [33] and Zimmerman and Vanderheiden [34] also stress the importance of quality assurance adding more specifically that quality assurance should be a continuous process in all phases of website development and should be applied to all developed applications, content and documents. They add external quality assurance as a factor that influences the implementation of accessibility standards during the design process. This adds the following two factors to the proposed research model:

• Internal quality assurance,

• External quality assurance.

Many authors specifically identify the knowledge of the workforce as an important factor in the process, specifically during the design and implementation phase [32, 33, 35, 36]. Nambisan and Wang [36] studied the effect of knowledge barriers on the adoption time of web technology. They conclude that the lack of relevant knowledge delays the adoption time of web technology even if they are potentially profitable to web development organizations. Although implementation of web technology has become much more commonplace since their study, the implementation of accessibility standards is still very new to many web developers. This study includes the knowledge factor to the research model:

• Knowledge The knowledge of the municipal stakeholder about the accessibility standards.

When a website is not built in conformance with accessibility standards, these can be addressed later. This is called retrofitting. Retrofitting accessibility standards at the end of the process is always more expensive than building it in from the start. As an example, [37] indicates that the cost of retrofitting can be as much as ten times higher. Boehm [37] describes the increasing costs when making changes at a later stage in the life cycle of a project.

In their research into the costs and benefits of applying accessibility standards, Velleman and van der Geest [38, 39] categorize the possible benefits of applying accessibility standards into four categories: (1) financial, (2) social, (3) technical and (4) legal/policy factors. All categories provide potential benefits.

Nambisan and Wang indicate that the knowledge of such benefits with the stakeholders is important [36]. They show that it plays an important role in the decision to adopt and implement the standards.

Examples of benefits of applying the guidelines early in the process could be reaching a larger audience, better performance (platforms, browsers, devices, etc.), faster, better findable in search engines, a better user experience and a contribution to corporate social responsibility [38, 39]. This study will therefore include them into the research model:

• Internal benefits;

• External benefits;

• Budget and cost.

Factors related to the organizational structure

The internal structure of an organization, but also the combination of different organizations involved in the adoption and implementation process can influence the adoption process. Thong [40] studied the adoption of information systems (IS) in small businesses and concluded that ‘While CEO and innovation characteristics are important determinants of the decision to adopt, they do not affect the extent of IS adoption. The extent of IS adoption is mainly determined by organizational characteristics’ (p. 187). One of those characteristics is the assignment of responsibilities. Folmer and Punter [41] indicate this in their list of factors for successfully implementing a standard. They argue that there should be a committed problem owner. This study adds this problem ownership to the research model:

• Assign responsibilities

Folmer and Punter [41] continue, ‘A dominant party or dominant process can greatly encourage adoption’ (p. 61). This is supported by Heuvelhoff and de Bruin [42] who show that involvement of network organizations can be a factor that considerably slows down decision making. Most adoption and implementation processes for accessibility standards include multiple organizations working together, causing problems when it comes to decision making and thus potentially slowing down the process. Heuvelhoff and de Bruin mention three risks of involvement of networks that could be interesting to consider for the adoption process of accessibility standards. Networks have a variety of actors with different characteristics, knowledge and interests. This can cause confusion during implementation. Networks can also be very closed, not letting in advice or comments from other external stakeholders. Finally, people in networks tend to be dependent of each other. This means that sometimes people in the network have to wait for others in the network to provide advice, approval, etc., before a decision can be taken. This study adds the three factors to the research model:

• Pluralism Different interests between stakeholders involved in the adoption and implementation process;

• Closedness Degree to which advice or comments from external stakeholders is accepted;

• Interdependencies Dependency on other people for advice, approval, content delivery, information, etc., influencing the continuation of the adoption or implementation.

Besides clear responsibilities within the process, also the quality of collaboration between municipalities can influence the process, as for instance, in the exchange of information and best practices between municipalities. Promoting interaction and interoperability between organizations helps to achieve accessibility [43] (p. 27) even if the collaboration is not much more than the exchange of data and best practices between different organizations [44]. This is why the study adds municipal collaboration to the research model:

• Municipal collaboration Collaboration and exchange of information and best practices with other municipalities to adopt and implement the guidelines.

Sometimes, there is a knowledge gap between the procurer and external partners who are mostly hired for their knowledge. Jensen [45] describes this ‘principal-agent relationship’ as ‘a contract under which one or more persons—the principal(s)—engage another person—the agent—to perform some service on their behalf that involves delegating some decision-making authority to the agent. If both parties to the relationship are utility maximizers, there is a good reason to believe that the agent will not always act in the best interests of the principal’ (p. 86). The main concern of external parties could be that they want to procure the order. The procurer should check this. Because this is a potential risk to the implementation of accessibility standards, this study adds these factors to the research model:

• Quality of procurement The degree to which the guidelines are mandated in the procurement requirements;

• Checking skills of outsourced party Checking the accessibility standards skills of contractors before contracting them.

With regard to the involvement in (strategic) decisions and degree and continuity of involvement during the adoption and implementation phase, Miller et al. [46] indicate that the role of stakeholders changes from one phase to another. They propose the political decision-making model. This study adds three factors related to the political decision-making model that are relevant. In the literature about web accessibility, managerial support is also shown to be an important factor [23]. This study adds the following factors to the research model proposed:

• Stakeholder influence and involvement The degree to which a stakeholder feels he can influence the adoption and implementation and his commitment to the adoption and implementation;

• Stakeholder responsibilities The extent to which the responsibilities of a stakeholder are clearly assigned in the process so the stakeholder and other stakeholders are aware of his responsibilities;

• Managerial commitment and decisions Commitment of management to implement the guidelines and to take steps if necessary to achieve that goal.

External factors

The Netherlands has different rules and legislations concerning accessibility for all citizens that could be applicable to websites. Hence, this study includes these as factors in the research model. This includes relevant rules and legislation from Europe and the UN:

• Rules and legislation Availability of rules and legislation requiring the adoption and implementation of accessibility standards (national and international).

Factors proposed by accessibility experts

In this study, structured interviews were conducted with experts to share their view on factors influencing the adoption and implementation. The main purpose of the interviews with the experts was to prevent gaps that could exist if only the input from the literature was to be used. To complete the research model, experts were selected with expertise and experience with regard to adoption and implementation of web accessibility standards in eGovernment. The interviewers did not show or discuss the output of the literature research with them. The experts included a web accessibility evaluation expert, an accessibility policy expert and a public procurement expert.

Using structured interviews, the experts were asked to identify factors that were most likely to influence the adoption and implementation process of accessibility standards in municipalities. The outcome of the interviews with the experts made it possible to add two new factors to the research model: (1) disability in circle, and (2) pride and ambition. Based on the interviews, ‘opinion on the guidelines’ was also added as a separate factor. The other factors mentioned by the experts were consistent with earlier findings from the literature except that it was decided to shift, combine and rename some factors as a result of the interviews. Checking skills of outsourced party is covered by quality of procurement and taken out as a separate factor, and legislation was split up into (1) legislation on accessibility, (2) other rules and demands and (3) citizen influence.

The two new factors mentioned by the experts could not be classified under the four categories that were established from the literature. Because they are related to personal experiences and feelings, they are grouped under a new category named ‘personal factors.’ The final model now includes 5 categories.

• 1. Adoption factors;

• 2. Factors related to the design process;

• 3. Factors related to the organizational structure;

• 4. External factors;

• 5. Personal factors.

Research model and coding categories

The result of combining the factors identified from the literature and the factors indicated by the experts can be seen in Table 1. All factors will be used as coding categories for the interviews with the stakeholders. This list is the starting point for further exploration with the stakeholders.

Table 1 The research model from the literature and experts that is used for further exploration with the stakeholders

Research design

This section describes how the adoption and implementation factors from the existing literature were validated to the adoption and implementation of accessibility standards of municipality websites.

After building a conceptual research model based on the existing literature (Table 1), semi-structured interviews with experts were conducted to identify perceived adoption and implementation factors. The two lists of factors were then compared, and factors that were additionally indicated by the experts were added into the research model (Table 3).

The research model was then validated using a comparative multiple case study as described by Yin (p. 13) [47] using semi-structured interviews with stakeholders from different size websites with different levels of accessibility. Answers were analyzed with the factors in the research model as coding categories. Based on this analysis, an experimental adoption and implementation model was constructed.

Case selection: Municipalities and stakeholders

This study examined municipality websites with different size (number of citizens) and level of conformance.

Dahl and Hansen [48] indicate that on the one hand, the size of municipalities is a significant factor when it comes to the implementation of standards and that on the other hand, the adoption of the guidelines can be influenced by the more complex organizational structure of a large municipality. For this reason, small, medium and large websites with different levels of accessibility standards conformance were selected.

To establish the conformance level, the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0 Working Group Note [19] with the Web Content Accessibility Guidelines 2.0 (WCAG2.0) [11] was used. These guidelines became a formal ISO/IEC standard in 2012 (ISO/IEC 40500:2012). The evaluation consisted of a full manual evaluation of the websites by experts of the Accessibility Foundation, an ISO 17020 accredited inspection organization for accessibility in the Netherlands. In this study, websites that are not conformant with the minimal set of guidelines for level A are rated ‘low,’ conformance with WCAG2.0 level A is rated as medium and level AA is rated as high.

Based on the literature, 18 internal and external stakeholders who worked within municipalities during the adoption and implementation process were selected.

The Dutch Ministry of the Interior and Kingdom Relations [49, 50] distinguishes three main stakeholders once the decision has been taken to implement accessibility standards: (1) the designer, (2) the developer and (3) the content editor. The KING i-Versneller [51], a support program instigated by the Dutch government to help municipalities adopt and implement accessibility standards, provides support for website project leaders, IT managers and advisors, web domain coordinators, implementation and support managers, content managers and communication advisors. The W3C/WAI implementation plan for web accessibility [52] advises to establish a coordination team within the organization and to look for representatives from key departments ‘such as marketing, web development, technical support, usability.’

The stakeholders selected from these groups and individuals are all people who can affect or are affected by the achievement of the organization’s objective as proposed by Freeman [53] and who indicate that they have an opportunity to influence the process as described by Rowley [25].

The final selection can be seen in Table 2 and includes internal decision makers, content editors, (external) web developers/designers, management and marketing staff from smaller and larger municipalities, both internal and external staff and with different levels of accessibility of the website at the time of the interviews. As key stakeholders, they have an important role in providing an explanation for the (lack of) adoption and implementation of web accessibility standards (as described earlier). While the intensity of the stakeholders’ involvement is different, they all indicate that they feel they had influence on the adoption and implementation process within the municipality. The goal was to recruit at least one internal and one external respondent for every municipality in the study. For smaller municipalities, this was less obvious because the relevant person was sometimes hard to reach. For two of the municipalities, the interviewer did not find an external staff member. In total, 18 persons have been interviewed.

Table 2 Overview of municipalities, the accessibility level of their websites and stakeholders and their position in the process

Procedure

The interviewer contacted the stakeholders by phone and e-mail and conducted semi-structured interviews with them on location, mostly in a meeting room of the municipality and, in case of external staff, on location in their office. The interviews were conducted in the period from May to October 2013. All interviews were recorded and transcribed.

During the semi-structured interviews, all stakeholders were asked questions that were relevant to their work situation (see ‘Appendix’ section). The questions were based on the subjects and factors identified earlier in the literature. There were 4 sets of questions related to: (1) management, (2) strategy and marketing, (3) design and development and (4) content and editing. If a stakeholder fell within more than one category, questions of all applicable categories were asked. This way, they were confronted with questions they were expected to be able to answer. If necessary, the interviewer asked the stakeholders to clarify their answers or to elaborate further.

The transcribed interviews were divided into episodes to make them more manageable. The factors in the research model received codes. Three research students from the University of Twente then acted as ‘coders’ and separately coded the interviews linking the answers of the stakeholders back to the factors from the research model.

Data analysis

The ‘coders’ assigned zero, one or more codes to the text transcription episodes. Because the large number of codes (40 codes) limited possible guessing chances, a Cohen’s kappa was not calculated, directly calculating the agreement in assigned codes instead. As there was low agreement in assigned codes between the coders (first round only 45 % agreement in assigned codes, second round even back to 41 %), extra coding rules to improve the results were agreed. The ‘coders’ agreed that the reason for the fallback in the second round was the complex content of one of the interviews. This was covered by a small set of extra rules for coding including the following:

• The codes for ‘knowledge’ must always be related to online accessibility;

• The codes for ‘complexity’ must always be related to accessibility standards and not to complexity of other standards or systems;

• The codes for ‘network structure’ must always be about situations inside the municipality;

• The codes for ‘procurement and communication with supplier’ should always be assigned when the supplier is mentioned;

• The codes for ‘rules and legislation’ that are not related to accessibility standards, use ‘other rules and legislation’, specifically added for this purpose.

The transcriptions were then analyzed using this improved coding scheme.

Results

For this study, 18 semi-structured interviews were conducted with stakeholders. In the coding process, the answers of the stakeholders were linked back to research model. For every occurrence of a factor during the interview, the coders ticked the assigned code.

Table 3 is an overview of the factors coded from the interviews with the stakeholders. The perceived level of influence is ordered as (1) no influence, (2) medium influence and (3) high influence.

Table 3 Overview of factors and their influence on the adoption and implementation of accessibility standards within municipalities

The category “high influence” indicates factors that are mentioned by multiple stakeholders and where stakeholders indicated that this was an important factor influencing the final result.

In the following paragraphs, the results are discussed per category.

Adoption factors

Which adoption factors are influencing the implementation of accessibility guidelines according to the stakeholders? Table 4 lists the number of quotes coded as adoption factors.

Table 4 Number of quotes coded as adoption factors

Not all factors from the models came forward in the interviews with the stakeholders. This is interesting because based on the literature and existing models one would expect these factors to have a significant impact.

Adoptions factors that were not named as influential by the stakeholders were:

• Relative advantage,

• Observable benefits,

• Network externalities,

• Related technologies,

• Current infrastructure,

• Communication channels.

Perceived complexity

The extent to which the accessibility standards are perceived to be easy or difficult to adopt and implement is covered in the factor “perceived complexity.” It is an indication of how the stakeholders perceive the accessibility standards. Stakeholders indicated that they experienced the accessibility standards as complex. Six of them referred to the amount of work necessary to implement the standards, and 10 indicated they would expect to have problems applying the standards to video on the web. Three out of the six municipalities have the perception that the standards concerning video are too strict. Some named the complexity of applying them to interactive maps, text and/or images.

Municipalities that have successfully implemented the standards perceive the standards as less complex than municipalities that do not yet meet the standards requirements. Also they are more positive about the standards. It is not always clear whether the perception of complexity is based on real experiences and a detailed study or knowledge of the standards, or an expression of hearsay.

Resources/technical possibilities

This factor describes the availability of information about the technical possibilities of accessibility including clear guidelines, explanation and toolkits that can facilitate the adoption and implementation. Some of the stakeholders looked the subject up on the web, and others followed professional training courses or asked experts for information. Information was also provided by the providers of content management systems (CMS) and IT systems and infrastructure for municipalities. Some CMS systems include a check for accessibility that prompts editors before the page is published. For example, if an editor forgets to add a description to an image, that image will be difficult to understand for blind people. The system will then prompt them and ask for input. Editors from 5 municipalities indicated this ‘warning system’ as a factor that influenced their implementation of the standards. This, however, does not automatically mean that the website will be accessible. Even systems that have this ‘warning system’ in place and where editors indicate this as a factor still have inaccessible pages with these same errors. Both editors and suppliers indicate that the problem causing this is not with them. Availability of resources makes it easier to adopt and implement the accessibility standards, though availability does not automatically lead to adoption and implementation. Stakeholders need to see the importance and their responsibility before they look for these resources.

Compatibility

Stakeholders indicate that compatibility influences the decision to develop a new website and/or to apply accessibility standards. Compatibility describes the ease to adopt and implement the accessibility standards with existing infrastructure or the existing web services. It is interesting to conclude that the factor compatibility is only mentioned by stakeholders from municipalities that have successfully implemented the standards. The municipalities that mentioned this factor have not only implemented the standard but also changed their web service orientation more toward the citizens. This change of orientation could be a reason for the increase in the perceived importance of accessibility standards. Sometimes the change of orientation led to the need for a new infrastructure, making the choice to adopt and implement accessibility standards easier.

Sponsorship

This describes the involvement of external agents with a degree of power on the internal adoption process. These agents include governments, programs, persons or other mandates. Five municipalities mentioned this factor. They named agents such as inspection/evaluation organizations, an authority like KING (Quality institute of Dutch municipalities), especially the iNUP program (National Implementation Program) as influencing the process. The iNUP program is one of the main sponsors of adoption and implementation named by all except one municipality. As this is a particular sort of legislation, it will be further explained when discussing external factors.

Web design factors

The interviews with the stakeholders were analyzed for all references to web design factors. Table 5 lists the number of quotes coded as web design factors.

Table 5 Number of quotes coded as web design factors

Knowledge

This describes what a municipal stakeholder (involved in the design process) knows about the accessibility standards. Within the design process, this factor received the highest number of quotations by stakeholders (79). Managers do not always have prior knowledge of accessibility before they start the process. The project leader of the largest municipality solved this by hiring an external accessibility expert and including accessibility standards clearly in the contract as a requirement and a responsibility of the suppliers.

The developers showed very different levels of knowledge. Some had an in-house expert, and some had built a website with a successful implementation of the accessibility standards before. The websites of three municipalities that scored a high level of accessibility were all developed by developers that (1) had successfully implemented the accessibility standards before and (2) had an internal accessibility expert.

Not all managers, developers and editors followed accessibility courses. Some managers hired an expert; some developers and editors asked their colleagues and read the guidelines themselves.

Stakeholders involved in the adoption and implementation of municipalities that have a low score on accessibility seem to have less knowledge of the accessibility standards.

Budget and costs

This factor addresses the planned and the actual costs incurred by a municipality on adopting and implementing accessibility standards. It is used both as a reason and as an excuse to adopt and implement the standards. Stakeholders in municipalities with a low score for accessibility indicated that the priority was lowered as a result of the available budget. They stated that when the project reached the end, it ran out of money, and accessibility standards were removed to a lower place on the to-do list. Stakeholders in municipalities with a high score for accessibility all had a special budget allocated to accessibility. One municipality with a high score for accessibility had to implement the standards after the website was developed and launched. The stakeholder describes that it took a lot of effort and extra costs to realize this. The interviews show that a higher allocated budget can facilitate the implementation. However, perceived benefits by management are a precondition. If management does not perceive sufficient benefits, priorities seem to move away from accessibility standards.

Internal and external benefits

The interviews show a list of comments about perceived benefits. Stakeholders describe what they perceive to be the benefits of adoption and implementation of accessibility standards. It is interesting that municipalities that score low on accessibility also seem to find accessibility standards less important. The webmaster of one website indicated that they have multiple channels to serve people and that people who are deaf can also visit the municipality if the website is not accessible.

Some comments about benefits include:

• The website is also accessible on mobile devices

• Show citizens that the municipality takes responsibility if necessary

• If all people are able to use the website, they do not have to call or visit in person. That may save costs (‘all’ includes people with disabilities)

• Citizens are more satisfied with the municipality (as it becomes easier to do transactions)

• The guidelines can be used as a weapon to convince other departments or convince the management to develop a new website

• The management of the website gets less expensive when the code is of good quality

• Having a faster website

• Having a better ranking in search engines

Stakeholders mentioned accessibility standards as an instrument to convince other departments and management to develop a new website. The reason they used this argument is not always primarily to adopt and implement the accessibility standards: One of the municipalities used this argument to develop a new website and then during implementation adjusted the development in such a way that it did not result in successful implementation of the standards. Nevertheless, stakeholders indicate the importance of (perceived) benefits.

Quality assurance

Stakeholders support the literature findings when it comes to quality assurance. Quality assurance helps monitor the degree of adoption and implementation of accessibility standards. They do require a certain level of knowledge and motivation with the people involved. Some municipalities in the study hired experts to support them in this process. Quality assurance helps municipalities in many ways including the possibility to save costs on (extra) external inspections. By continuously monitoring the activities of their suppliers, they can prevent implementation problems in an early stage of the process, thus saving expensive repairs at a later stage.

Organizational structure

The interviews with the stakeholders were analyzed for all references to organizational structure. Table 6 presents the number of quotes coded as factors related to organizational structure.

Table 6 Number of quotes coded as factors related to organizational structure

Quality of procurement

This describes the degree to which the guidelines are mandated in the procurement requirements and checking the accessibility standards skills of contractors before contracting them. All stakeholders indicate that accessibility standards were part of the requirements in the request for tender that was sent out. This is remarkable because only two of the municipalities did finally implement the guidelines. Stakeholders indicated that in the talks with potential suppliers, they all claimed to be able to implement the accessibility standards as required. During the process, it seemed that this was not always true. Municipalities did not always check these claims by the suppliers. Also, many stakeholders believe that the technical part of the website is conformant with accessibility standards, while this is not always true. This belief is mostly based on the suppliers/developers claims which are not checked for validity. Only one municipality asked the supplier to repair the non-conformance after they had the website checked.

Importance and priorities

During the interviews, the stakeholders mentioned importance and priorities as key factors in the process of adoption and implementation of accessibility standards. Importance and priorities play a central role in almost all the factors, having the highest co-occurrence with other codes (98). Therefore this factor was added to the coding list during the coding process.

Assign responsibilities

This factor refers to the extent to which the responsibilities of a stakeholder are clearly assigned in the process so the stakeholder and other stakeholders are aware of their responsibilities. The interviews show that if responsibilities are not clear, persons involved start pointing to suppliers or to others persons from inside or outside the municipality for accessibility. Although in all municipalities one or more stakeholders are responsible for the website and the implementation of accessibility standards, none of the stakeholders interviewed indicate they felt a particular responsibility for the overall implementation of the standards. For instance, some stakeholders pointed to the built-in features of CMS systems, although they were sometimes not aware that these features do not cover all standards. Suppliers indicate that they never received a question about expanding the features to cover the standards. They did not take that responsibility themselves. One municipality employed an external project leader with the task to reach a high conformance level for the new website. This project leader indicates that it took much effort to check and push suppliers.

Many internal municipality stakeholders indicate that they did not have to report to their management.

Managerial commitment and decisions

This describes the commitment of management to implement the guidelines and to take necessary steps. This factor was mentioned by stakeholders in all disciplines. In municipalities where managerial commitment with the website and online services was high, stakeholders perceived fewer problems with adoption and implementation.

Non-management stakeholders play an important role in the commitment of managers. Managers indicate that they gave higher priority to implementation of accessibility as a result of advice or pressure from web and communication colleagues in the municipality. In the largest municipality, the stakeholders indicated that the subject was high on the priority list of the new website process thanks to the continuous interest of a municipality council member. This continuous interest created a feeling of urgency that helped keep the subject high on the agenda during the process.

In municipalities with a lower score on accessibility, stakeholders mentioned that they had to discuss with the manager many times to keep the subject on the managers’ priority list. Some indicated that management was in the end not interested in the subject but used it nevertheless as a reason to develop a new website.

Municipal collaboration

This factor refers to collaboration and exchange of information and best practices with other municipalities to adopt and implement the guidelines. Stakeholders indicate that this collaboration can help reduce the cost of implementing accessibility standards, thus also facilitating the decision to adopt and implement. The developers of the largest municipality in the study set up a partnership with dozens of other municipalities. The stakeholders of that municipality experienced collaboration as a factor that determined the successful implementation of accessibility standards. They shared the cost with the other municipalities.

However, collaboration is not always an indicator for successful implementation. One of the municipalities selected to use an open-source CMS and joined a network of municipalities using this CMS. In this network, they shared techniques and modules including accessibility functions. Nevertheless, the end result of their work did not include a good implementation of accessibility standards.

Pluralism

Different interests between stakeholders within a municipality involved in the adoption and implementation process. This factor seems to have a negative influence. Four municipalities mentioned pluralism. Examples include departments having very different wishes with regard to the amount of information, interaction or reading levels on the website, but also differences of opinion about tasks and restrictions of the website. Moreover, departments may have different views on the adoption and implementation of accessibility standards.

Interdependencies

This factor refers to the dependency on other people for advice, approval, content delivery, information, etc., influencing the continuation of the adoption or implementation. Interdependencies seem to have a negative influence: One example includes having to wait for text documents provided by other departments that have to be made accessible first.

Closedness

Closedness addresses the degree to which advice or comments from external stakeholders are accepted. This factor was not indicated by stakeholders to be very influential.

Personal factors

Interviews with the stakeholders were coded for personal factors as displayed in Table 7.

Table 7 Number of quotes coded as personal factors

Stakeholder influence and involvement

Stakeholders indicate that they are committed and can influence the adoption and implementation process. Almost all stakeholders found that they could influence the process (at least within their own department or task). As found earlier with the factor responsibility, most stakeholders believed that their website was conformant with accessibility standards. When confronted with the fact that this was not always true, the stakeholders of the municipalities that score low on accessibility pointed out that this was not something they could have influenced. When asked for details, they pointed to other departments or stakeholders.

Opinion on guidelines

The stakeholders have very different opinions about guidelines. They all agree that the guidelines are necessary and ‘a good thing.’ Of the municipalities that find that some of the guidelines are too strict (5), most perceive the guidelines about accessible video requirements as a problem. Their solution was sometimes even to remove the videos from their website or transfer them to other websites such as YouTube. Particularly municipalities that score low on accessibility mention the strictness.

Pride and ambition

Municipalities that successfully implement the accessibility standards are proud of what they accomplished and mostly ambitious to do more. On the other hand, municipalities that have not successfully implemented the standards perceive the standards as complex, less important and too strict.

Disability in circle

Knowing how people with disabilities use the web helps to understand the importance of applying accessibility standards. Stakeholders in 3 municipalities had this experience and support this factor for that reason.

External factors

The number of quotes made by stakeholders regarding external factors was analyzed. Table 8 presents the number of quotes coded as external influences.

Table 8 Number of quotes coded as external influences

Legislation on accessibility

In the Netherlands, there are many laws and regulations that directly or indirectly require accessibility. Mostly, this is in the form of equal opportunities for all citizens. One of the most important is a high-level agreement between all government agencies in the Netherlands that they should have finished the implementation of accessibility standards by the first of January 2015. This is described in the i-NUP government program [6].

This factor is related to the “sponsorship” factor. Five municipalities mentioned legislation aspects when talking about sponsorship; specifically the i-NUP program (National Implementation Program) was referenced as influencing the process. At the time of conducting this study, three of the municipalities in the study had reached the required conformance level (level AA).

Stakeholders indicate that legislation influences the perceived importance and the priority in the process. Stakeholders in municipalities that already conform with the standards are very positive about legislation, while stakeholders within municipalities that score low on accessibility perceive this legislation as complex and unnecessary. Stakeholders remark that other rules and legislation may compete with accessibility legislation and lower the priority. This can have a negative influence on the adoption process. Also, they remark that there is not yet a sanction for that. The interviews do not prove that legislation is a decisive factor though it is important in helping to convince other stakeholders and some stakeholders indicate that it is a factor that helped in placing the subject on the list of priorities.

Other rules and demands

Stakeholders from three municipalities mentioned the importance of other rules and demands that are not directly accessibility related but compete for a high placement on the priority list. These include rules for online security, privacy and even formats and other regulations. The vast number of rules and demands that can also apply to their websites could make it hard for small municipalities to meet all demands. This perceived competition between legislation, rules and demands can cause accessibility to move down on the priority list.

Citizen influence

One expert already indicated that complaints by citizens put the subject on the agenda so they started working on the subject. Stakeholders from three municipalities also mention this factor as influential. Two municipalities made changes following complaints by citizens. When citizens do not complain, municipalities may wrongly presume that this means they have correctly implemented accessibility standards and thus not carry out further work on implementation. This indicates that citizens can influence the adoption and implementation. The interviews show that if citizens would complain more, this would have a positive effect on the adoption and implementation of accessibility standards.

Conclusions

This study proposes a new adoption and implementation model (Fig. 1). The model is based on a combination of factors identified in the literature and from semi-structured interviews with experts and stakeholders. Existing adoption models from the literature were adapted for the adoption and implementation of accessibility standards within municipalities.

Fig. 1

Model describing factors that influence the adoption and implementation of accessibility standards within municipalities

An important conclusion is that not all factors from the adoption models in the literature were mentioned by the stakeholders in the study as influencing the adoption of accessibility standards within municipalities. Factors that were not mentioned as influential include: relative advantage, observable benefits, network externalities, related technologies, current infrastructure, communication channels and closedness. This is interesting, because based on the literature and existing models one would expect these factors to have a significant impact. The only factors from existing adoption models that were mentioned by the stakeholders as influential are: compatibility, complexity, sponsorship and resources.

The experts and the stakeholders in this study added a new category of ‘personal factors.’ The stakeholders acknowledged the personal factors ‘influence & involvement,’ ‘opinion on guidelines’ and ‘pride & ambition’ as having a high influence on the process.

Stakeholders also mentioned the perception of ‘importance and priorities’ as an important influence on the adoption process. At the same time, the ‘importance and priorities’ of accessibility standards within municipalities impact the influence of factors during the implementation phase. When the coders were asked to include it in the coding, it rendered the highest co-occurrence score (98) with the other codes in the coding process making it the central most important factor in the model displayed in Fig. 1.

The top 10 factors influencing the adoption process of web accessibility standards within municipalities are:

• 1. Assign responsibilities

• 2. Knowledge and experience

• 3. Managerial commitment and decisions

• 4. Perceived benefits

• 5. Opinion on guidelines

• 6. Legislation on accessibility

• 7. Pride and ambition

• 8. Other rules and demands

• 9. Compatibility

• 10. Sponsorship

(11. Citizen influence)

The top 10 factors influencing the implementation of accessibility standards are:

• 1. Selection and procurement of external supplier

• 2. Assign responsibilities

• 3. Knowledge and experience

• 4. Quality assurance

• 5. Perceived complexity

• 6. Budget and costs

• 7. Municipal collaboration

• 8. Technical possibilities

• 9. Pluralism

• 10. Interdependencies

Recommendations

The following is a set of recommendations for the adoption and implementation of web accessibility standards in municipal websites proposed as an outcome of the present study:

• Assigning responsibilities at all levels helps both the adoption and the implementation of accessibility standards within municipalities. Responsibilities should be clearly spelled out, as the study shows that while stakeholders claim to be in full control over their part of the process, once confronted with non-conformance of the end result, they pointed to others for responsibility.

• Responsible agents within municipalities should acquire or hire the necessary knowledge to check processes and claims for which they are responsible. Within the design process, the factor “knowledge” received the highest number of quotes. As found earlier with the factor “responsibility”, most stakeholders believed that they had influence on the process indicating they had sufficient knowledge for their task even though this was not true. Stakeholders involved in the adoption and implementation of municipalities with a low score on accessibility also seem to have less knowledge of the accessibility standards. This lack of knowledge may also explain the high perception of complexity of the standards within municipalities with a low score on accessibility.

• Management should be committed to implement the guidelines and to take all necessary steps. This can be organized by other stakeholders. In municipalities where managerial commitment with the website and online services was high, stakeholders perceived fewer problems with adoption and implementation. Managers indicate that they gave higher priority to implementation of accessibility as a result of advice or pressure from council members, but also of web and communication colleagues in the municipality.

• Although legislation, rules and demands do not directly guarantee the proper implementation of accessibility standards, they do have a positive influence on the process.

• The study shows that it is important to not only include the standards and skills requirements into the procurement, but also organize quality assurance to check/validate the claims made by the suppliers before, during and after the process.

• All stakeholders indicated that it was clear from the beginning of the process that the website would have to comply with accessibility standards. This requirement was often included into the requirements for both internal and external suppliers, but the responsibilities to check them should also be clearly assigned within the organization.

• Budget should be clearly allocated for the implementation of accessibility standards. Stakeholders in municipalities that have a high score for accessibility all had a special budget allocated to accessibility.

• For a good implementation, it is necessary to make sure that all stakeholders (including external suppliers) have sufficient knowledge of the web accessibility standards. Stakeholders involved in the implementation process in municipalities that have a low score on accessibility seemed to have less knowledge of the accessibility standards.

Discussion

Many organizations are working on the implementation of accessibility standards. However, they often seem unable to fully adopt or implement them, even if they are actively pursuing it. This study presents a new adoption model that aims to better help local governments in the identification of factors influencing the actual adoption and implementation of web accessibility standards. The model is based on the literature and interviews with experts and stakeholders.

The study has a limited scope and is therefore difficult to generalize. This means that the main outcome of the study is a new adoption model that can be used as a basis for further research and extension to assess this model and to establish a more scientifically tested model.

The new model can be potentially interesting for a broader application than just accessibility standards, such as for the adoption and implementation of privacy and security standards and open-source standards. Future research could focus on these applications and also extend the amount of websites and stakeholders included. The model may also be interesting for the implementation of online payment requirements, citizen service-oriented policy and other rules and demands that organizations have to apply to their websites and applications.

It would also be interesting to validate the new adoption model in different countries, as there can be additional cultural or other differences influencing the process.

The new adoption model was built in part on the existing adoption models identified in the literature. This study extended these models to the adoption of Internet standards. However, the outcomes show that the existing adoption models were only partly relevant to the adoption of accessibility guidelines. Nevertheless, they formed a good basis for the new adoption model.

There seem to be important differences in the implementation of accessibility standards between small and large municipalities. Although the number of municipalities in this study is limited, the differences are visible in the data. Some factors influencing the adoption and implementation of accessibility standards may need different strategies in small or large municipalities. This will merit from further research.

Appendix: Interview questions

Below are the questions for the structured interviews carried out with the stakeholders. The questions are derived from the subjects and factors that were treated in the literature. If necessary, the interviewer can ask the stakeholders to clarify their answers or to elaborate further.

See Table 9.

Table 9 Questions for the semi-structured interviews carried out with the stakeholders