Keywords

1 Introduction

The communication can be considered a set of signs that refers to behaviors that occur among of two or more persons and which provide a way to create meanings between them. The term alternative and augmented communication (AAC) is defined as other forms of communication beyond the oral communication, such as the use of gestures, sign, facial expressions, use of alphabet boards, graphic symbols, use of sophisticated computer systems with synthesized speech and others [6]. Thus, communication is considered as alternative when the individual has no other form for communication, excepted oral communication, and it is considered augmented when the individual has some form of communication, but this is not enough to maintain communication links or establishing social exchanges. The purpose of this project is the development of an alternative tool using interactive media and expanded interaction with touchable and voice communication devices in Portuguese. This tool can provide a significantly difference to a person’s life that has any kind of restriction or is unable to communicate, especially those with boundaries regarding communication, such as difficulties and speech limitations.

This paper is organized as follow: concepts and contextualization are presented in Sect. 2. Section 3 presents our prototypes and the development process of the prototypes. Finally, the Sect. 4 presents the conclusions and future works.

2 Concepts Involved and Contextualization

Cards and boards with graphics symbols are used as features to alternative and augmented communication. This collection of images used in cards and graphical boards shares common characteristics and is designed to address different requirements or needs of all kind of users with disabilities in communication. One of the most used symbolic systems in the world is the PCS (picture communication symbols), created in 1980 by an American speech therapist named Mayer Johnson [1]. The PCS system has the following characteristics: clear and simple designs, easy recognition, suitable for users of any age. The system is easily combined with other figures and photographs for creating individualized communication resources.

There is software based on PCS system that allows communication of individuals with pre-recorded or synthesized voices. A voicer is an electronic recording/playback resource that helps people to communicate. Using these devices, the users can express their thoughts, feelings and desires by choosing and pressing an appropriate message that is pre-recorded on the device. Messages are accessed by keys which are associated with images (pictures, symbols, pictures) or words that correspond to the recorded sound. However, the interface of voicers is not suitable for many people, especially people with physical disabilities or other conditions that act directly on the human brain. Furthermore, the use of overlapping images on keyboards creates some limitations based on usage of images, symbols and words.

Thus, the aim of this project is to develop an application of alternative and augmented communication to be used to any user with an appropriate interface, allowing the user to have the ability to communicate and become part of society. During the study we defined a set of users and what type of application they need, because we know that different users need different interfaces: a user with autism needs a different communication tool if we compare with one user with cerebral palsy, for example [4, 7]. After our first prototype implementation, intended for a specific type of user - with cerebral palsy, other interfaces were implemented, allowing the use of application by a greater number of users, also with cerebral palsy but with different limitations, fitting the special needs of each user. Probably, in the near future, more interfaces will be required.

3 Experiments and Prototypes to Supporting Alternative and Augmented Communication

Prototypes are graphical representations, not necessarily functional, of a system (or part of a system) in the design phase. They can be models, drawings and screens designed to simulate user interaction with a system [5]. According to [3], build prototypes is a very good way to explore ideas for a project before investing time and resources in its implementation. It is sort of test and consolidates ideas using cheaper resources and its used have a fast development process. Prototyping is an effective technique to test hypotheses and check the user’s needs in early development phase, when it’s quite cheap making changes in the design [2].

In this project the prototyping allowed us to design and evaluate the IVA tool and realize during the requirements gathering the needs of design another interfaces, with new forms of interaction, since users have different characteristics and needs for interaction.

3.1 IVA Prototype – an Alternative and Augmented Tool

The prototype of IVA tool was designed to enable the communication of patients with cerebral palsy that live in a hospital in Brazil and educational psychology professionals, as a tool to support therapy sessions, allowing patients to express themselves. We also included some functionality that support some activities performed during the therapy sessions.

Figure 1 shows the layout of the main interface of the developed tool. The logo is available at the top left of the screen; the navigation buttons are located at the top right: they allow access to basic functions of the tool: eat (a), drink (b), play music (c), play games (d), how I feel (e) and speak (f).

Fig. 1.
figure 1

Home screen of IVA prototype

Full size image

The user can choose the button/option EAT (as showed in Fig. 2) and select the possible food menu - all options were suggested by professionals involved and the images were chosen to fulfill what patients know and used in their daily activities. Clicking on the image, the prototype emits a sound corresponding to the text of the selected button and the sound is repeated every time an option is selected. The same procedure occurs for other areas: Drink, Play Music, Play Games and How I Feel.

Fig. 2.
figure 2

Option EAT detailed

Full size image

The Speak button is under development, and it will propose a simplified alphabetic keyboard, so the patient can type complete sentences, to express what they think, feel or want. The numeric keypad, displayed on the homepage, was a request of the educational psychologists to allow patients with skills and mathematical knowledge, use the numeric keypad to make small calculations - this functionality was already used during some therapies with numeric cards of paper and was incorporated into the IVA prototype. It’s important emphasize that in the main interface as well as other tool’s screens were designed by the professionals already involved with the patients and the functionalities can be inserted, removed and changed according to the needs of patients and professionals.

During requirements collecting and initial tests with patients, we noticed that some patients were more affected by hands and arm movements. Thus, a new functional prototype was created for patients with cerebral palsy with reduced control of hands and arms and they are no able to use touch, or use conventional mouse, or pointers.

3.2 Adaptations and New Forms of Interaction

The new prototype has interaction mechanism with bigger objects and reduced amount of options, but enables communication of the patients with reduced control of arms and hands, and combines concepts of multimedia and communication boards, allowing a visual and voice interaction between the patient and professionals involved in the pedagogical therapies.

This new interface consists in a set of images and sounds to be exposed to the user and the interaction is through a click on an adapted mouse (two separate boxes simulating a conventional mouse, with one left and one right sensors), which was called “boxes clicks” – Fig. 3.

Fig. 3.
figure 3

Boxes clicks

Full size image

The new part of the prototype has a fully visual interface; the screen can display three images, as shown in Fig. 4, with the middle image is highlighted (see Fig. 4) and is the image selection. The left image is the option that was previously presented and the right image is the next option being displayed. For each image or interaction performed, the corresponding sound is presented referred to the image/interaction selected, in order to stimulate the hearing and oral communication of the patient.

Fig. 4.
figure 4

Main new interface

Full size image

The user, by hitting the left side of the “boxes clicks” is choosing that s/he wants to select and/or execute the highlighted option, and the prototype provides the contents of the selected option and triggers the corresponding sound. By hitting the right side of the “boxes clicks” another option appears highlighted: the next option. The options are arranged on a carousel, which is repeated at the end of each set of options. The choice of an option of the carousel allows patients to participate in therapies with technological resources. In the initial tests with users, some chose to use the “boxes clicks” with feet and a new adaptation is being prepared to make it more resistant.

4 Conclusion and Future Work

We present prototypes developed to support the process of communication between therapists and patients with little or no oral communication during the therapy sessions in a hospital.

Other existing tools were tested but they did not meet the needs of these users - most have multiple disabilities and the tested tools do not provide specific support for cognitive difficulties and motor difficulties at the same time. Even our prototypes have been adapted for each user and many of the users appropriated the offered resources on their own way.

We understand that people with multiple disabilities need specific technology to support communication and that these resources have be tailored to each user. Our prototypes are constantly evolving and adapting to serve patients in their therapy.