Keywords

1 Introduction

Deaf people have been at an unfair disadvantage with regards to education, employment and access to technology [1]. This is due to the fact that limited interactive technologies address usability and accessibility concerns, stemming mainly from the literacy-related barrier they experience. Accessing content on the Internet is problematic for many deaf people since as pupils, many leave school with severe reading and writing inadequacies [2]. Equally important is the fact that the first language for many deaf people is the sign language of their country and not the oral one [2, 3]. Providing information in sign language can alleviate these types of barriers and impact positively in the integration of deaf people into the IT society [2]. In an attempt to make deaf people full citizens, the European Union of the Deaf (EUD) declares three objectives [1]: recognition of the right to use sign language, the use of communication and information for empowerment, and equality in education and employment. Support for the right to use sign language is recognized in international and European legal documents too, such as the Brussels Declaration on Sign Languages in the EU (2010) and the United Nation Convention on the Rights of Persons with Disabilities (2006) [2].

Using websites and online services requires a level of literacy and technology competency. Literacy issues and content not being presented in sign language are just two reasons that prevent many deaf people from having positive user experiences when accessing technology. Several past projects that have investigated such issues are DictaSign, eSign, ViSiCAST and the South African Sign Language (SASL) machine translation project of Stellenbosch University [4]. The need for more research that can lead to the design and development of more accessible and usable IST for deaf users is stressed [2]. A definitive description of published manuscripts in the field of IST for deaf users in the discipline of HCI that focuses on accessibility will contribute in classifying topics that new researchers can undertake within the field. In support of this classification, this paper will address the following objectives:

  1. 1.

    Construct a map of existing research topics in the field of IST for deaf users in the discipline of HCI that focuses on accessibility

  2. 2.

    Summarize the purpose of each code category of the map

  3. 3.

    Identify the least and most researched topics of the map

2 Method

A validated and existing systematic approach was applied to construct the map of IST for deaf users in the discipline of HCI that focuses on accessibility (see Fig. 1).

Fig. 1.
figure 1

The process adopted for the construction of the map

Full size image

The approach has been applied to other areas before, such as to develop web design guidelines for the elderly [5] and to construct a map for the field of Computer Assisted Language Learning [6]. Figure 1 presents the process supporting the approach that was adopted to construct the map for this study. A more detailed discussion on each stage of the process and how it was considered in this study follows:

Stage 1: Build the Corpus. According to [6] it is in this stage that a corpus of literature for the particular research field must be determined. This corpus will be thoroughly reviewed in order to construct the map for the field under investigation. The corpus in this study included 69 manuscripts that were published from 2000 to 2013 and which had a specific focus on IST for deaf users. All main sources of HCI literature that focuses on accessibility were studied. The small number of manuscripts can be considered as an indication that this area of research is understudied. Four main research sources devoted to the field of HCI with a focus on accessibility were reviewed to select the manuscripts for inclusion into the corpus. The manuscripts were extracted from two journals and two conferences. These included the journals’ of Universal Access in the Information Society (UAIS) and the ACM Transactions on Accessible Computing (TACCESS). Conferences that were reviewed are the ACM CHI Conference on Human Factors in Computing Systems and the ASSETS conference. The distribution of manuscripts from the four sources is presented in Table 1. The corpus does not include introductions to special issues or editorials.

Table 1. Distribution of manuscripts based on source
Full size table

Stage 2: Literature Overview and Initial Coding Scheme Development. According to [6] it is in this stage that an overview of the built corpus, which was determined in stage 1, is conducted to elicit basic themes and to develop a coding scheme with code categories. The overview is based on reviewing the title, keywords and abstract from all the manuscripts of the corpus. To elicit the basic themes, an initial overview of the corpus’s manuscripts was conducted by the authors. The overview was based on extracting codes from the title, abstract and keywords of each manuscript. The output from this stage was the identification of 31 keywords, which represent the basic themes discussed in the manuscripts of the corpus. The initial coding scheme was consequently developed, consisting of 31 code categories.

Stage 3: Refinement of the Initial Coding Scheme. According to [6] it is in this stage that a focus group session is conducted to verify, expand or limit the initial coding scheme that was created in stage 2. A selection of manuscripts from the corpus are reviewed, discussed and classified within existing or new code categories, which are created if required. The outcome from the focus group session is a revised coding scheme. A focus group session was conducted to refine the initial coding scheme that consisted of the 31 code categories, which were extracted from the initial overview conducted in stage 2. Three independent HCI and accessibility experts participated in this focus group session. The experts were required to randomly select twelve manuscripts (17 %) from the indexed corpus. Time was provided to the experts to read the title, abstract and keywords from a selected manuscript. After overviewing a manuscript, consensus-based discussions followed between experts and the moderator, who was facilitating the discussions. The consensus-based discussions focused on the classification of a selected manuscript into an existing code category or if necessary into a new code category that would need to be established during the focus group session. If classification could not be determined or consensus not be agreed by reading the title, abstract and keywords of a selected manuscript, its introduction and conclusion sections were then also reviewed. This process was followed twelve times, in order to classify each selected manuscript into a code category. A total of 2 new code categories were added during manuscript classification, increasing the total number of code categories from 31 in the initial coding scheme to 33 in the revised coding scheme. The revised coding scheme was the next topic of discussion. In order to proceed from the revised coding scheme in this stage to the construction of the first version of the map in the next stage, the 33 code categories had to be organized into logical structures. The construction of logical structures for the code categories was likewise determined in the focus group session with the experts. Code categories were required to be divided into categories and subcategories. The division of code categories was imposed by the data (manuscripts). Subcategories are defined when differences with other subcategories in the same main category are apparent.

Stage 4: Construction of Map Version 1. According to [6] it is in this stage that the first version of a map is constructed. The structure of the map is based on the revised coding scheme, which was the outcome of stage 3. By constructing logical structures from the code categories of the revised coding scheme in stage 3, it is now possible to construct the first version of the map in this stage. The first version of the map for this study consists of 7 main categories, 22 subcategories and 4 second-level subcategories. The map must be organized in a manner that complies with two criteria: internal homogeneity within the generated categories and external heterogeneity among categories [6]. Reference to code categories in the focus group sessions includes main categories and all levels of subcategories.

Stage 5: Refinement of Map Version 1 and Construction of Map Version 2. According to [6] it is in this stage that the first version of the map, which was created in stage 4, is revised and assessed for its meaningfulness and accuracy. It is therefore necessary to work back and forth between the manuscripts and conduct a content analysis to ensure that all manuscripts can be assigned to the map’s categories/subcategories (the logical structures of the code categories). In addition to title, abstract and keywords that were reviewed in stage 2, it is also required to comprehensively review the introduction, conclusion and future implications (if any) sections of each manuscript during this stage. An important factor contributing to classification is saturation; the classification of the corpus manuscripts into code categories without incongruity [6]. The outcome from the refinement of the first version of the map is map version 2. Based on results collected from stages 3 and 4, refinements were made to the first version of the map. Meaningfulness and accuracy of categories and all levels of subcategories for the first version of the map were further examined by classifying all the corpus manuscripts within these. As previously mentioned, it was necessary to work back and forth between the manuscripts and the map. A content analysis that included a review of the introduction, conclusion and future work (if any) sections of all manuscripts was conducted. Following the content analysis, each manuscript had to be classified into a single code category of the map. Therefore, in order to assign a manuscript to a single code category, it is imperative to consider the objective of the manuscript. This provided a means to determine its main focus and to likewise reach saturation, by classifying all manuscripts of the corpus into a single code category. The second version of the map is the outcome of refinements done to the first version of the map (see Table 2).

Table 2. Refinements during map construction
Full size table

Stage 6: Evaluation of the Proposed Structure and Inclusiveness of all Categories in Map Version 2. According to [6] it is in this stage that the card sorting technique is applied to independently cross-check the code categories of the second version of the map, which was created in stage 5. Furthermore, it also assists in the procedure of implementing new refinements to the map (if necessary). A new panel of experts will participate in the card sorting technique. Disagreements on classification (if any) are resolved by discussing classification differences and identifying the purpose and contribution of the manuscripts until consensus can be reached. The second version of the map was evaluated by applying the card sorting technique. In addition, to ensure the validity of the results, a second focus group session was also conducted to further refine the second version of the map on the basis of cross-checking and reflective discussions. Frequency tables were also applied to summarise the experts’ classifications as frequency counts and percentages. They are the simplest method for representing categorical and ordinal data. They are commonly used as exploratory procedures, with an attempt to establish how the different categories of values are distributed within the sample. Similar to the first focus group session that was conducted in stage 3, three new independent experts in HCI and accessibility were participating in this stage too. They were required to randomly select fifteen manuscripts (21 %) from the indexed corpus. The classification of the selected manuscripts into code categories proceeded. Following classification, experts shared insightful opinions regarding the revised map and the classification of selected manuscripts within it. These were largely influenced by their research background and expertise in the field of HCI and in the area of accessibility. Consolidation and consensus was reached through meaningful discussions in several cases where different opinions and classifications surfaced. In addition, results from the frequency tables also supported the course of determining the final classifications of the selected manuscripts and likewise supporting the reflection of these refinements into the third version of the map. Including the author’s classification of the selected manuscripts with those of the three independent experts, an overall percentage agreement of 71.6 % is reported for manuscript classification. This percentage is determined by measuring each participant’s classification for a selected manuscript against the consolidated and consensus-based classification (if required), which was the outcome of discussions. Simply stated, classification differences were ultimately resolved by means of discussions that were supported by the results indicated in the frequency tables. It entailed discussing the reasoning motivating each expert’s classification for a selected manuscript in alignment with the manuscript’s objective. Once agreement was reached regarding classification, consensus and saturation is likewise achieved. A more in-depth analysis of the frequency tables indicates that for 33.3 % of the selected manuscripts evaluated there was 100 % agreement regarding classification. For another 33 % of the selected manuscripts evaluated there was 75 % agreement regarding classification. For the remaining 33 % of the selected manuscript evaluated there was 50 % or less agreement regarding classification. It is for this set of manuscripts in particular (50 % or less agreement) that more extensive and meaningful discussions were conducted in order to reach consensus regarding classification. The third version of the map is the outcome of refinements done to the second version of the map, based on the results from the evaluation that was conducted in this stage (see Table 2). The final map (version 3) is presented in Fig. 2.

Fig. 2.
figure 2

IST for deaf users in the discipline of HCI that focuses on accessibility - Map version 3

Full size image

3 Results

Results are discussed in terms of the 3 paper objectives mentioned in the Introduction.

3.1 Objective 1: Construct a Map of Existing Research Topics in the Field of IST for Deaf Users in the Discipline of HCI that Focuses on Accessibility

Table 2 summarizes the refinements that were made to the map from its initial first version until its improved third version (see Fig. 2). Map version 2 represents refinements that were based on the outcomes from the first focus group session, conducted in stage 3 of the process. Map version 3 represents refinements that were based on the outcomes from the second focus group session, conducted in stage 6 of the process.

In Table 2, Category integration occurs when an existing code category (main or sub) is integrated into another existing code category. A Rename occurs when the name of a code category has been rewritten. When a subcategory has been removed from the map it is listed in the column Removed subcategory. Reclassified manuscripts represent those manuscripts that had to be reclassified into different code categories in comparison to those that they were initially classified in. It must be noted that the total number of reclassified manuscripts does not relate specifically to the manuscripts that were selected for evaluation in the focus group sessions alone. The reclassification also occurs from refinements made to the actual map versions. For example, when a subcategory has been removed from a map version it is necessary to reclassify the manuscripts that were included in the removed subcategory into a different code category (main or sub).

3.2 Objective 2: Summarize the Purpose of Each Code Category of the Map

HCI literature that focuses on accessibility was reviewed to develop the map. This section provides an overview of its main categories, highlighting key issues covered in their subcategories. It is notable that subcategories classified within a main category can consist of a limited number of manuscripts, yet are regarded worthy of individual separation based on topic, need and prospect of attracting new researchers.

Assistive Technologies. The main category of Assistive Technologies focuses on tools and software that can assist deaf users experience technology in a more positive and usable manner. It supports them in performing tasks that were formerly difficult to accomplish or not accomplishable. This is possible due to technology enhancements or due to the alteration of interaction methods with technology and software. Assistive technologies must promote social inclusion, autonomy, independence and life quality. This main category includes six subcategories. The first subcategory focuses on captions. These are real-time textual alternatives for audio, which can also include descriptions of sound. They are regarded as the most common type of assistive technology used by deaf users since they transform aural into visual information [3]. The second subcategory focuses on tooltips, which are used to make icons more comprehensible. Deaf users prefer visual information since they are visual learners; therefore tooltips can have a positive effect. The third subcategory focuses on video. As an assistive technology, video is significant for deaf users. Video for mobile communication is mainly discussed. The fourth subcategory focuses on teletypewriters. Teletypewriter (TTY) technology enables deaf individuals to have direct and equal access to emergency call centers. Users can type, send and receive messages for communication. Regarding the corpus, this subcategory had a poor representation of manuscripts. Nevertheless, it presents potential for new research. The fifth subcategory focuses on icons. This entails investigating the design and use of iconographic touch interfaces on mobile applications. The sixth subcategory focuses on Computer Assisted Translation (CAT), also referred to as Machine Translation (MT) or Generation systems. Their aim is to provide deaf users with the capabilities to translate text/voice of vocal languages into sign languages and to produce, modify and review online avatar-based sign language contributions. It requires expertise in several related topics, including sign language linguistics and synthesis to create system vocabularies, knowledge-based sign synthesis architecture [7], and sign language animation and recognition to present manual and non-manual means of expressions in sign language translations using avatars. This subcategory is further divided into four second-level subcategories that are interrelated; it is however possible for research to be conducted in a single second-level subcategory only. Each of these second-level subcategories can be highly technical components. The first second-level subcategory is editor/scripting software, which is an additional capability to intervene and correct translations that are error-prone. The second is sign language linguistics and synthesis that focus on the study of sign language as a natural language. This can include its history, structure, acquisition of language, morphology, syntax, semantics and pragmatics. Examining the linguistics is required to ensure to the best extent possible, the linguistic adequacy of sign generation tools and to support robust conversions [7]. Sign synthesis is methods/techniques that are applied to provide fluid mediums of sign language, for conversation synthesis, by converting sign language from a stored and textual medium. The third is animation and avatars that are used to display sign language translation representations to the users. The synthesis of signed speech/text can therefore be portrayed through the movements of a human signing figure. In order to provide translations, an avatar is required to display the synthesized sign language. The fourth is sign language/gesture recognition that mainly focuses on the technical approaches implemented to capture and recognize sign language gestures. In broad-spectrum, gesture recognition utilizes mathematical algorithms to interpret human gestures.

Universal/Inclusive Design. The main category of Universal/Inclusive design focuses on the design of services, products, software and environments that are usable and accessible for as many people possible, despite their differences in abilities, gender, education, age or cultural backgrounds. The aim is to provide equal opportunities for people to participate in economic, social, technological, cultural, recreational and entertainment activities. With regards to the IST map, this main category discusses manuscripts that emphasize Universal/Inclusive design in general. In overview, apart from the manuscripts categorized with the main category, two more specific subcategories are determined. The first subcategory focuses on policy/guidelines. Policies are intended to inform stakeholders and policymakers about the adaption and use of technology by people with disabilities. Guidelines and acknowledged standards (e.g. W3C, ISO) intend to guide developers in the design of accessible software [8]. The development of guidelines and the transformation of technical specifications into international standards are also discussed in this context. Policies and guidelines pertaining to the Web are specifically identified. Hence, the Web is defined as a second-level subcategory. This second-level subcategory is further divided into five third-level subcategories: vlogging, hyperlinks, forums, sign language dictionaries and collaboration/communication tools. Vlogging investigates the use of video technology and techniques by the Deaf community to post video content on websites. Alternative forms of hyperlinks are compared and new forms of hyperlinking, which are based on video material that enable browsing without written language, are discussed. Forums empower online participation and deliberations on various topics. Video is used as the communication medium and forums can accommodate several user types, such as advanced contributors’ and lurkers. Sign language dictionaries provide online translations for spoken and sign languages. The interfaces of such web dictionaries must be designed applying user centered design methodologies and it is required that composition rules of signs be encoded. Collaboration/communication tools are browser based systems that support face-to-face communication between deaf and hearing members of a team.

The second subcategory focuses on research methods. The objective of manuscripts categorized within this subcategory is to either provide insights into new methods for conducting research with deaf participants or to modify existing research methods in a manner that accommodates the needs of deaf users. Henceforth, two second-level subcategories have been created within this subcategory; new and adapted/guidelines. Example of a new method is a remote testing technology to conduct user studies in sign language at a lower cost or the use of drawing software with set stories to conduct evaluations with deaf children [9]. In the adapted methods/guidelines, it is acknowledged that impairments can impact how researchers will design questionnaires, user interviews, focus group sessions and user evaluations to elicit more reliable and valid data.

Educational Software. The main category of Educational Software focuses on using technology software to teach or self-learn. Software, web tools, mobile applications, games and e-learning environments are the types of platforms discussed within this context. Learners include deaf and hard-of-hearing individuals, as well as hearing parents of deaf individuals. Providing tools that can improve the literacy levels of deaf children and students who are native signers of a particular sign language is the main focus.

3.3 Objective 3: Identify the Least and Most Researched Topics of the Map

In Table 3, the total distribution of manuscripts for each of the three main categories is presented. In order to identify the least and most researched topics within each of the main categories, the total number of manuscripts for their subcategories are also provided. Regarding Table 3, it must be noted that manuscripts can be classified into main categories, without necessarily being classified into a subcategory. Second- and third-level subcategories are not included in the table.

Table 3. Distribution of corpus manuscripts into code categories (main and sub) of the map
Full size table

It can be concluded from the results presented in Table 3 that emphasis is being devoted mainly to the CAT/MT/Generation systems subcategory. It is seen as a crucial area for deaf accessibility, particularly over the web. Despite it being the most researched topic from the corpus however, it is relatively unexplored considering the sources reviewed to create the corpus and the time period that was covered. The Universal/Inclusive design main category is likewise crucial and unexplored. Contributions in this category in the form of policies/guidelines and new research methods will be most valuable to designers and developers so that they can provide deaf users with products and services that are accessible, useful and provide positive user experiences. Researchers will also benefit by knowing how to implement different and the most appropriate research methods with deaf users, ensuring that the participants have been properly considered. The data collection process and analysis of the collected results will also have increased validity. In terms of the educational software main category, research is needed to provide deaf people with solutions that can assist them in learning and improving their sign language and vocal language skills. Work conducted in the CAT/MT/Generation systems subcategory will prove invaluable in this regard. It is also important to remember that people who are not deaf, particularly parents of deaf children, require solutions as well to improve their sign language skills in order to communicate with their children. This is critical because the longer it takes for deaf children to start communicating with their parents; the more their language modality is affected.

4 Conclusion and Future Work

This paper presents an inventory of research into IST for deaf users in the discipline of HCI that focuses on accessibility, on the basis of four relevant sources. It provides guidance to new researches who are interested in conducting research within this landscape. In addition, the paper attempts to stress the need for more research activity in this discipline. This need is voiced by considering the number of deaf people worldwide that are required to use technology that is not accessible and usable for them. Noteworthy is the limited number of sources available discussing IST for deaf users in HCI literature that focuses on accessibility. The limited number of manuscripts from these sources that were relevant for the review over a period of more than ten years, further support the call for more research activity.

A systematic approach based on a six-stage process was considered to conduct the inventory. The process assisted in achieving the three objectives of the paper. The first objective was to present a map of existing research topics in the field of IST for deaf users in the discipline of HCI that focuses on accessibility. This was achieved with map version 3 (see Fig. 2). The second objective was to summarize the purpose of each code category within the map. This is achieved in Sect. 3.2, which synthesizes the findings of the map. An overview of the type of research conducted in each main category and all their levels of subcategories is provided. The third objective was to identify the least and most researched topics of the map. This is achieved with Table 3, which lists the distribution of manuscripts from the corpus based on main and subcategories.

Considering the map for IST for deaf users in the discipline of HCI that focuses on accessibility, future directions have been identified. Researchers can position themselves in this landscape and contribute to new work in the identified areas. Addressing the problems of non-accessible technology and the educational and employment disadvantages that deaf people experience are areas that require further and immediate exploration. The authors’ future work will be directed towards improving web accessibility for deaf users. In particular, efforts will be directed towards the construction of a new research method that can support HCI experts and developers when evaluating and respectively designing websites for deaf users. Based on the map for IST for deaf users in the discipline of HCI that focuses on accessibility (see Fig. 2), such work would be classified into the Universal/Inclusive Design (2) main category. Within this main category, it is further classified into the Research methods (2.2) subcategory. As a second-level subcategory it is classified into the New (2.2.1) subcategory, since it will provide a new method of evaluation that is specifically for the deaf user group.