Keywords

1 Introduction

Today, Information and Communication Technology (ICT) is becoming increasingly available to people around the globe, especially in developing countries. Smaller devices such as tablets and smart phones are available for increasingly cheaper rates, and are reaching all groups of people. There is increasing interest by researchers and NGOs in applying computers, mobile phones, and tablets for serving economically poor populations in the field of Information and Communication Technology for Development (ICT4D). Technology has proven to be an effective tool in alleviating poverty, in areas such as education, networking, and employment.

Vocational training and skill development are connected to both economic productivity and social wellbeing [1]. TVET helps people to gain employment, especially those with limited formal education [2]. However, it is difficult to make TVET readily accessible to economically and educationally challenged communities, at first simply for its novelty, but more importantly because most technology has been designed for formally education, literate, and urban populations [3]. ICT driven TVET is a promising solution to accelerate learning for people with low literacy, as compared to conventional models because it is more easily adapted to multimedia instruction, but should be designed with the end user in mind. ICT elements like multimedia textbooks were found to be advantageous in reducing average learning time and in enhancing understanding of a subject matter when presented properly [4].

The User Interface (UI) of the ICT devices for TVET, need to be designed with the level of literacy of the target users in mind. The design process needs to be re-thought for the ICT4D context, where many of the basic assumptions that underpin the methods may not always hold true for low literate community. For example, many illiterate users are unfamiliar with the concepts of lists and the hierarchy used in modern UIs [5]. This paper shall discuss orality and its roots, as well as abstract versus operational thinking and its implications for ICT4D [6].

This paper will present the observations and results of a series of usability studies on low-literate, rural audiences using two units from a vocational training course for toilet construction in five villages in India. The tests were conducted to measure the efficacy of the package which included “pre-training” activity, contextual videos, and a supporting game module. Five testsFootnote 1 were conducted at different villages across the country from 2016 to 2017, with 60 illiterate and semiliterate participants. In the consecutive tests, problems were observed and analyzed at two different levels: user and the system level. Some solutions for these problems were included in the next trial of the test. The study discovered various issues related with the package we tested, and refinements were made iteratively on the instructional video, and design changes were incorporated into the UI of the game module. The paper also presents a pre-training module as a solution that empowers low literate rural audiences to more easily take up ICT for using vocational training. The module enables low literate users to physically use the devices, trains the user to listen to and answer questions, and introduces and familiarizes the user with the UIs and its components from the start.

2 Related Works

Introducing ICT into TVET has accelerated learning in populations with low- or semi-literacy. Bhavani et al. has demonstrated that using ICT in vocational education for rural populations is more flexible, accessible, and provides a high amount of scalability [2]. From the same study, it has been observed and analyzed that there are advantages to using interactive multimedia textbooks over standard formal school textbooks. ICT becomes a powerful medium for providing TVET, especially for the rural sector, when compared with the traditional classroom setup designed for urban areas.

The target audience for ICT4D includes illiterate or semi-literate populations. For this reason, ICT4D researchers have proposed doing away with text-based instruction, and have instead recommended non-textual designs [7]. In this section, we shall discuss the various modalities available or introduced to overcome the issues faced by low-literate or semi-literate users, especially understanding and using abstract symbols and following navigation procedures. UIs that use voice, graphics, and video have been proposed to better meet the requirements of the target population. Audio-output as a means of instruction (as opposed to text) has become primary topic for research, as it is a more natural means of expression well suited to learners who may struggle with literacy (ibid). Multimodal interfaces such as the VideoKhethi, introduced by Cuendet et al., a mobile system, helps farmers to find and watch agricultural extension videos in their own language and dialect, have been developed for use on smart phones with graphics and touch interfaces including speech recognition. This was designed to help improve agricultural practices among rural groups [8]. Exploration of touch gestures to enable a better and natural experience for first-time; low-literate users have been conducted. Research has demonstrated that soft-key mapping on devices with keypads is more challenging for low-literate users [9]. According to the same study, ICT devices, like tablets, exploit the advantages of touch gestures, and needs no or comparatively less training for first time users, when compared with text driven devices like laptops or mini-computers.

Oral users are not necessarily acquainted with the system of abstract learning from books or other formal systems. Rather, learning often happens in the original environment, in concrete situations, and through practical experience. This means that learning and teaching happens in an informal, yet very immersive way where oral and visual transmission of knowledge is the primary mode of delivery. With this in mind, it is important to properly characterize the target audience. Sherwani et al. tries to better understand ICT4D users in terms of their thinking, communication and culture [10]. In their work, the illiterate community is not considered as less privileged, but referred as “oral users.” Orality talks about how people think, communicate, and learn in a culture where writing has not become prevalent. Orality theory claims that writing has so fundamentally transformed consciousness in literate cultures that we—literate researchers—are unable to grasp how oral cultures and people function. When one develops a user centered design for oral users, consideration should be given to cultural practices of community knowledge building and transmission which includes: (a) organization and transmission of information, (b) learning information, and (c) remembering information.

3 Our Work

Our team is a South India-based organization, concentrating on developing technology enhanced vocational and skill development courses for empowering women, especially in villages and rural areas across the country [3]. We provide vocational courses using ICT devices, such as tablets and laptops, reinforced by expert trainers and post-training support. The educational model we use is a virtual media enhanced vocational course, which has been developed end to end by the team at AMMACHI Labs. The vocational training model follows a multimodal learning approach in which the user learns through verbal and nonverbal (audio, visual and textual) representations of the material. In order to test the efficacy of our approach, we took two units from our Rural Toilet Builder vocational course, focusing on training rural women in India to build toilets for their community.

3.1 Target Community

Usability tests were conducted at 5 different villages across the country from 2016 to 2017, with illiterate and semiliterate women participants (see Table 1). All of the women were from families that are financially supported by men and/or women working for daily wages, and in unskilled, manual work. Over 50% of our participants had little or no literacy, and less than 15% could read complex sentences. To test the level of literacy, participants were given a sheet of paper with some sentences on it in their mother tongue, and requested to read out what they could. As for the participants’ familiarity with technology, mobile phones are the most common technology that these women may have seen or handled. None of them had previous experience in using tablet or a computer. The primary language for communication was Kannada, Tamil, and Hindi (depending on the location). Low-literate users think, communicate, and learn in a culture where writing has not become internalized. In order to make them eligible for a TVET model, the user were first introduced to the material through a physical training where they used physical gestures to answer questions. The training was taken to the next level by asking question through images and answering through picking the correct image. After this the same set of questions were asked through tablet devices and answered by touch-gestures, slowly equipping them to use technology-enhanced models for learning.

Table 1. Overview of the series of user studies
Full size table

3.2 Experimental Setup – Application

The teaching application was driven by an animated character that is similar to a “traditional” village lady. This means she was wearing clothing similar to what our target population wears (sari), has a similar skin, hair, and eye color, wore common accessories (no fancy gold ornaments, etc.), and behaved in ways that were familiar to them. The language of instruction was also their mother tongue. A character narrator was introduced to the users so that they could more easily connect to the application in a grounded way, rather listening to instructions through a disembodied voice. The interface was designed with minimal text elements, and supported by graphics, animation and audio intelligible to a person who cannot read [11].

3.2.1 Classroom and Interactive Games

The classroom provides the theoretical knowledge of the vocational trade the student is learning, and the necessary concepts for the eventual occupation they are training for. Each classroom contains multimedia content: videos displaying recorded demonstrations and explanations by experts in the field, accompanied by animations, images, and some text. It gives theoretical knowledge on a specific component in the course through video footage, accompanied by text, audio, 2D and 3D illustrations and animations. In the toilet building course, the video included real women from a village in Tamil Nadu recorded in handling the different elements of construction. This is an important part of our training videos, as it boosts the confidence of the target students to see people they can identify with doing these new, sometimes difficult skills. The course also includes simple, interactive games in between classroom chapters that use 2D illustrations and animations to create interactive exercises where users navigate through practical, real-world scenarios and apply the theoretical knowledge gained through the classroom. This module requires the user to answer questions, played by audio, on the content provided in the instructional video. The response is given through a touch gesture, and is succeeded by a feedback screen which shows the correct response. This exercise is repeated to re-emphasize the concepts to the user. The user interface does not provide negative reinforcement for wrong responses from the user, in order to avoid reducing the student’s enthusiasm; rather, it guides the user to the correct answer in case they make an error.

3.2.2 Practical Assessment

Following the classroom and intervening games, the user’s understanding is assessed. The units we selected for the usability tests required both cognitive and operational skills, and the assessments reflect both. The student was assessed in an environment that allowed them to use the skills they were learning, and which was conducive for assessing cognitive skills. Cognitive skills were assessed with oral questions by the field staff on concepts taught in the training. Operational skills were assessed by asking the student to physically demonstrate the proper use of the tools explained in the classroom in the real world, where they are observed using the tools.

3.3 Experimental Setup – System Setup

The vocational training courses, and our usability tests, require a classroom set up to deliver the educational content to the participant. In terms of location, both urban areas and rural communities in India frequently lack both steady electricity and quiet, undisturbed spaces for maintaining a classroom environment. Rural areas are frequently difficult to reach and face a lack of facilities including secure rooms and poor internet connectivity. Our TVET design already addresses some of the environmental and social challenges that working in a rural, remote area bring, such as limited electricity, etc., through our innovative deployment model discussed at length in our other publications [3]. However, even after building local trust, providing training in low-infrastructure environments and encouraging enthusiasm from users, we still faced the issue of the users not fully understanding the original UI.

3.4 Iterations on Design Principles, Ethnographic Issues and Solutions Incorporated

The first and second usability study came up with issues which we have classified as:

3.4.1 Educational

The participants were mostly illiterate, first-time users of technology. The teaching methodology and the UI which was initially used were not effective enough with the target population. Many students failed to understand the concepts in the training or even how to use the UI. The primary challenge faced by participants was to think and consider concepts in abstract terms. This could be due a lack of formal schooling and exposure to abstract ideas in a formal setting. This is reflected in popular learning and development theories which argue that abstract thinking is learned mostly through formal, modern education models [12]. For example, user was able to understand the usage of the tool, i.e., a plum bob to check the level of the surface. But their knowledge was only restricted to checking the level of the bricks which was introduced to them, not to other materials. The users were also not familiar with a model of education where they needed to pay attention to a video, process the knowledge, and then apply it in the successive sections. Even for games, which were slightly easier to grasp, the participants struggled to understand that the games were meant to test concepts from the associated video classroom. The participants typically answered from their existing knowledge and life experience, even if it contradicted what was just taught in the classroom.

3.4.2 Interface Related Issues

3.4.2.1. Classroom

In the original classroom, the animations and the video footage used abstract symbols like a tick mark, crosses, and concentric circles to indicate correct or wrong procedures and to bring attention to a particular portion in the visual. This was initially not understood by the users, as it was the first time they were introduced to illustrations and animations using these abstract symbols. The video was narrated in the regional language, but followed a more formal style that did not include some the regional terms or slang they were comfortable with. This caused the participants to misunderstand some topics discussed in the video. Following the initial user tests, the animations in the video were redesigned to be more contextual, and less abstract for better understanding in the successive tests. The operational skills were re-emphasized by repeating the instruction on proper usage of tool in the video. Some of the important changes in the iteration we made included changing the camera angles for the user so that they felt they were a part of the process, using close and tight shots for validation scenarios, using expressions and hand gestures to convey correct and wrong procedures, and included some consequences of doing a step in the wrong manner (see Fig. 1a and b).

Fig. 1.
figure 1

(a, b) Comparing the initial version and final version of Plumbob video

Full size image

3.4.2.2. User Interface

The original interface tested used buttons and abstract symbols to operate and navigate through the application. The concept of “navigation” was new to them. The application was navigated using buttons which used minimum text in the regional language. Abstract symbols such as ticks or cross marks were used as icons to give feedback to the user on a particular question. The game modules in this initial test introduced help videos in the introduction screen of each unit. This did not help the users much because this population is not familiar with the concept of “help videos”. The user played the games with touch gestures, and each answer was followed by a feedback screen which showed the correct response. This was repeated to re-emphasize the concept to the user. A correct response was appreciated by the animation character by gestures to motivate and increase the engagement of the user.

3.4.2.3. Adjusting the UI and Introducing a Pre-training Module

In addition to adjusting the symbols used and perspective of the games and videos based on the first usability study, we designed and tested a “pre-training” module in the latter half of the usability studies. This was meant to help acquaint the students with the user interface and components of the training so that they would be more comfortable, and therefore successful in the training. The pre-training module includes a set of educational activities to develop the students’ ability to understand different question templates and to understand and become familiar with the common elements used in the training. The pre-training session was mainly divided into: (a) Physical representation, (b) Paper prototype, and (c) Tablet interface. Each of the three sessions had all of the variations of questions in the interactive game module. The templates included: True or false, identify the parts of a particular object, sequencing steps, and answering by listening to a context and choosing the correct option from a set of options. In the first session, we demonstrated the symbols using our arms and hands, and asked the participants to reproduce the gesture, as shown in Figs. 2 and 3. All of the participants responded correctly with the new gestures. This equipped them to gain the knowledge of abstract symbols like right and wrong for answering a yes or no question.

Fig. 2.
figure 2

Physical tick gesture

Full size image
Fig. 3.
figure 3

Paper prototype

Full size image

In the paper prototype session, we introduced the abstract symbols that are used in the course through images on paper. Large “Yes” or “No” symbols (tick and cross) and the user interface were first introduced on paper. The participants were asked to touch the icon on the paper as their response to the set of questions. The same set of questions which was asked in the first and the second phase of the pre-training were reproduced on the tablets. While the participants seemed to grasp the abstract symbols and progression of the activity, there were still some issues in using the technology itself. Specifically, it was difficult for the participant to correctly understand the touch and the drag gesture, often applying too much pressure, not understanding the consequence of using too many fingers, or not touching for the appropriate length of time.

3.5 Quantitative and Qualitative Results

The above graphs (Figs. 4a and b) show the results of Study 2 and Study 5, which are two comparable user studies. Study 2 was the first time the challenges by students were fully documented, and by Study 5 all of the improvements to the UI, including the pre-training, were tested. The results from studies 3 and 4 were helpful and provided more insight to us on what we could improve, however results are not usable for comparison cases in this analysis. The results are shown in terms of the participant’s literacy levels: For the non-literate group, there is a dramatic change from 0% of questions being successfully answered, to 80% successfully answered. 30% of the questions were answered without help from the moderator. For the low-literacy and literate groups, there is also a notable shift. For low literacy participants, the percentage of questions answered without any help goes from 25% to 63%. During later studies the participants were noticeably more confident and less reluctant to use the system. These results are encouraging for future studies that can assess the learning of populations in other areas and in other contexts. The limitations of this study, including a limited population, on testing a single course (toilet building), and short time duration could be addressed in future work.

Fig. 4.
figure 4

(a, b) Rate of successfully answering questions

Full size image

4 Conclusions

The results of comparing Study 2 and Study 5 (our two comparison cases) show a strong increase in the ease with which the non-literate students could engage with the tablet-based training. Contextualizing the content and introducing pre-training module helps students succeed because it gives them practice in using the interface to answer quiz-style questions. This approach demonstrates an initially effective strategy to address the challenges faced by illiterate and first time technology users in accessing vocational training. To date and using this model, we have successfully trained over 250 women who have collectively built over 250 toilets in 21 states across the countryFootnote 2.