Keywords

1 Introduction

The use of technology in education has led to the development of many applications and Web-based solutions aimed at supporting learning in a creative and amusing environment. Many applications have been designed as digital games to encourage and engage students of any age in their learning process. Unfortunately, for students with visual impairments the opportunities are very limited, because there still exists a significant gap in the accessibility of serious games and applications for learning.

In the literature some specific educational games or applications suitable for visually impaired children have been proposed, such as BraillePlay [1], a suite to teach the braille code. However, a visually impaired student has a very limited number of potential applications to consider, and they are for very specific topics. Accessibility guidelines have been proposed in the literature for several years [2] to require the design of applications suitable for all users and not specific for a certain disability. Nevertheless, this field still requires further studies and research to overcome that gap.

Most of the interaction modalities used for educational games greatly exploit the visual channel to present and display educational content in order to allow students to learn them by imitation. Unfortunately, this approach is not suitable for children with visual impairment. Therefore, we must propose equivalent, accessible and adequate alternative solutions for those who cannot see.

This means a great challenge for all those development teams that must develop a system for people with visual disabilities, because in most cases they do not know the tools and techniques appropriate to generate systems that meet the necessary requirements based on the usability and accessibility criteria that these users need.

In this paper we present a methodological proposal to build interactive systems using tangible user interfaces and gamification techniques. Taking as a case study the teaching of geometry concepts to children with visual impairment. The methodology has been designed to be simple and practical for all those development teams that have the responsibility of creating systems in which the blind child can interact in a simple and natural way, therefore, the interaction of the systems must be based primarily on touch and sound.

Our use case is based on the implementation of the proposed methodology to develop a system that meets the necessary quality requirements that is capable of teaching geometric subjects in children with visual problems.

The rest of the paper is organized as follows: Sect. 2 presents some related background. Section 3 describes the methodologies analyzed. Section 4 presents the proposed methodology. Section 5 presents the implementation of the methodology and finally, Sect. 6 presents the conclusions and future work.

2 Background

Since the METUIGA methodology follows the user-centered design principles and aims to develop interactive systems that include tangible interfaces and gamification techniques, concepts of these themes that compose it are shown below.

2.1 Tangible User Interfaces as Learning Tools

A tangible user interface (TUI) can be defined as one in which the user is provided with a physical representation of the digital information, allowing the user to literally grasp the data with their hands. This is possible thanks to the fact that the data is matched with the physical representations. A TUI must provide feedback to the user, either through the touch itself, through the digitized objects we refer to, or in aural or visual way when the interaction with that object has ended [3].

With a TUI we can transcend the common human-computer interaction, which is usually performed with a screen and images in two dimensions. This interaction can move on to the three-dimensional plane, making the user interact with something closer to reality.

Interest in tangible interfaces has been growing since the early nineties. One of the pioneers in the world of tangible interfaces is Hiroshi Ishii, head of the Tangible Media Group at MIT, who began to investigate with this type of person-to-person interaction in the mid-nineties. The new idea of Tangible Bits arose with the aim of joining the physical world with the digital one. The first tangible interfaces were created where objects, surfaces and spaces were used to materialize digital data [4].

Since the emergence of Tangible User Interfaces (TUI), numerous applications with different functionalities have been created, but hardly any of them has been applied in educational settings. Several reasons why TUI can improve the learning framework [5] are the following:

  • Use of physical materials. Keeping in mind that perception and knowledge are linked together, then manipulating physical objects will make it much easier to assimilate their nature. For example, three-dimensional objects can be understood more easily if presented physically than in a digital form.

  • Possibility of engaging the user. Interacting with TUI is much more natural than any other method of interaction, therefore it can be more engaging and accessible for children with disabilities.

  • Very useful for collaborative learning. Applications with a tangible interface can be designed to be collaborative, enabling several users to interact with the same objects at a time, in contrast to conventional software applications in which a single user interacts with a single screen.

  • It has a greater potential as a learning method when dealing with certain topics. As for example when studying the structure of molecules in chemistry or in biology. It has been shown that relying on 3D figures is more useful than relying on drawings or illustrations.

  • For all these reasons TUI are a promising technique when it comes to educating any child, but for blind children in particular, they contribute with even more benefits. Only the fact of enhancing their tactile capacity is already a giant step, since, as mentioned, this is essential for their further development. In addition, through an application based on TUI they could get to know the aspect and shape of very large objects by recognizing them in a tactile way, as we can put these objects to scale to be manipulated.

For all these reasons TUI are a promising technique when it comes to educating any child, but for blind children in particular, they contribute with even more benefits. Only the fact of enhancing their tactile capacity is already a giant step, since, as mentioned, this is essential for their further development. In addition, through an application based on TUI they could get to know the aspect and shape of very large objects by recognizing them in a tactile way, as we can put these objects to scale to be manipulated.

2.2 Gamification as Learning Tools

Gamification is based on the use of videogame design elements in non-game contexts to make a product, service or application more fun, attractive and motivating [15]. Thus, [16] raises gamification as the use of games’ own designs and techniques in non-playful contexts in order to develop development skills and behaviors. In this context, our gamification approach refers to the application of game mechanics to areas that are not properly game, in order to stimulate and motivate both competition and cooperation between players [17, 18].

For the most part, the authors agree to point out gamification as a fundamental factor in increasing user motivation. To motivate is to awaken the passion and enthusiasm of people to contribute their abilities and talents to the collective mission [15]. So, if you want to use gamification techniques, you need to know the keys to motivation to design games that engage different types of players as we will see later [19]. In this way, gamification techniques are breaking into organizations in order to enhance the motivation and commitment of employees and customers. The fields of use range from innovation, marketing, talent management and learning, to the development of healthy and responsible habits [19].

In this context, the fundamentals of gamification according to [20] are dynamics, mechanics and components. The dynamics are the concept, the implicit structure of the game. The mechanics are the processes that cause the development of the game and the components are the specific implementations of the dynamics and mechanics: avatars, badges, collection points, rankings, levels, teams, among others. The interaction of these three elements is what generates the gamified activity as presented in Fig. 1.

Fig. 1.
figure 1

Pyramid of the gamification elements [20].

Full size image

In the educational context, gamification is being used both as a learning tool in different areas and subjects, as well as for the development of collaborative attitudes and behaviors and autonomous study [21]. In fact, it should not be seen as much as an institutional process but directly related to a contextualized teaching project, with significance and transforming the teaching-learning process [22].

In this way, gamification can favor all these wishes of the students through the different mechanics and dynamics of the game, but as they point out [23], it is very important that there is a controlled relationship between the challenges shown to the students and the ability of these to carry them out, because if a challenge is too easy, it will cause boredom in the student, while an unattainable challenge will result in frustration, concluding both options in a loss of motivation for learning, being the rewards a very important aspect of Gamification.

2.3 User-Centered Design

The user-centered design (UCD) is an approach that consists of putting the end user at the center of the product design and development process and covering the entire life cycle of the product, that is, from the initial planning stages and analysis of requirements until final validations.

UCD methodologies are based on various international standards, such as: ISO 13407: Human-centered design process, ISO9241-210 standard for the human-centered design of interactive systems, etc.; and define a generic process that includes a series of activities throughout the product development life cycle, but without specifying the methods to be used in each of them.

One of the central ideas proposed by the UCD methodologies is that development processes cannot be resolved as linear processes, but require iterative and agile reviews, which involve constant reviews and evaluation of processes throughout the entire process. Life cycle of solution development.

3 Methodologies Analyzed

There is no formal methodology for interactive system design using tangible interfaces and gamification techniques much less related to children with visual problems, where user aspects are analyzed in a context such as education. For this reason, we carry out an investigation of different methodologies that have been proposed and that have been used for the design of interactive systems, with the purpose of identifying those characteristics that can help in our methodological proposal.

3.1 MPIu+a

The MPIu+a methodology [6] is oriented towards the design of user-centered interactive systems. The proposed model has the following phases: requirements analysis, design, implementation, launch, prototyping and evaluation.

One of the important aspects of the proposal is to integrate software engineering with usability and accessibility engineering principles, providing a methodology capable of guiding development teams during the process of implementing a specific interactive system. The methodology has a color coding oriented to software engineering identified with the color blue, the prototyping includes techniques that will allow the subsequent evaluation phase and is identified with the color green and the evaluation with yellow color, who includes and includes methods of evaluation.

3.2 LEGADEE

In 2012 a methodology called LEGADEE (LEarning GAme DEsign Environment) was proposed [8], it is a useful tool to help design educational games. The objective of the methodology is to facilitate collaboration between the different actors who must intervene during the design of a game.

The methodology is composed of several blocks that correspond to the sequence of the phases that represent the general process of creating the game. In the middle, all the elements external to the project involved in the creation of the game, as domain experts, designers and end users. The Workforce corresponds to the different actors participating in the project, here the different roles to participate (pedagogical experts, psychologists, developers, designers, among others) are described and selected. Teams refers to the computer tools that will support the different actors for the creation of the game. Materials is related to documents, database and any artifact used directly or indirectly as material to create the game.

The methodology includes 7 phases: customer needs, specification of pedagogical objectives, conception, quality control, performance, evaluation with the client and use/maintenance.

3.3 DOODLE

DODDLE (Document-Oriented Design and Development of Experimental Learning), is a methodology proposed [7] for creating serious games focused on documents. This model was influenced by the Analysis, design, development, implementation and evaluation phases of the ADDIE model. The 4 stages involved in the creation of serious games are: Analysis, design proposal, design documentation and production documentation. In turn, each stage is supported with the output of three documents.

To validate the model, the authors proposed new students to use for the conception of a serious game. After the observations obtained, they have noticed that the vocabulary it provides is necessary to be able to communicate between the different roles. In addition, DOODLE has obtained a positive effect on the optimization of production time, educational and playful quality of the game.

3.4 Deficiencies of the Methodologies Analyzed in Relation to the Purpose of the Proposed Methodology

A methodology is important for the design of interactive systems for children with visual problems, since it involves the participation of different experts, such as: students, teachers, programmers, among others with the purpose of defining objectives applied to the context of use. This should take into account two complementary objectives, a tactile objective in order to offer tools that can be manipulated for the child and provide feedback as it progresses, as well as progression scenarios and a recreational objective, with the interest of offering a favorable environment of learning, where different aspects are taken into account, such as: challenges, punctuation, rewards, rules, among others.

Each of the analyzed methodologies (MPIu+a, LEGADEE, DODDLE) has been applied to a context of educational use with a higher education student or researchers. This indicates that user aspects for children with disabilities have not been involved in the initial phase, such as: cognitive, academic, age, gender, learning styles, among others. These aspects may vary, that is, for children with visual impairment, aspects may differ compared to a hearing child. This leads to the importance of a detailed analysis that allows identifying limitations, behaviors, preferences, among others, in such a way that it serves as support to detect the specific needs for each child.

On the other hand, the methodologies analyzed are aimed at a type of user without disabilities, which is why they do not cover the expectations we seek. In addition, there is no methodology for the production of interactive systems that use tangible interfaces and gamification tools, indicating the importance of proposing a methodology that meets these characteristics.

4 Proposed Methodology

Based on this analysis, a methodology called METUIGA (MEthodology for the design of systems based on Tangible User Interfaces and GAmification techniques) is proposed, focused on the development of interactive systems for users with visual problems following the principles of user-centered design and The iterative cascade life cycle will also have tools to use gamification techniques and tangible interfaces in the project.

METUIGA is organized in stages that determine the stage of development in which we are (Fig. 1). The scheme clearly reflects, with a color coding based on the Mplu+a methodology [6], four basic concepts:

  • The attached tools for the correct use of gamification techniques and tangible interface tools (Blue).

  • Software engineering processes of the iterative cascade life cycle (Red).

  • The processes in which tests related to the results obtained in the software process must be implemented (Yellow).

  • The processes that must obtain a satisfactory assessment by the development team and end user (Green).

4.1 Considerations of the Elements in the Proposed Methodology

The following elements are part of the development process, but its objective is to support the main phases of the development process (Analysis, Design, Implementation, Launch).

Users

In current development models, designers and/or programmers decide for users, choosing metaphors, organizing information and links, choosing menu options, among others. A User-centered Design process should make it clear that this is the case only by looking at the scheme the first time. This is what is reflected by having users on the left side covering the rest of the stages of the entire process, in this case, since the user is someone with visual problems it is of great importance that they have contemplated from early stages in the development process.

Gamification

The gamification module works in parallel with the analysis and design phases in order to guide the development teams in the correct selection of the mechanics and dynamics to be implemented based on the needs presented by the user in the project.

Tangible User Interfaces

The tangible user interfaces module provides the necessary tools for the development team to be able to implement this type of technology. Providing a construction scheme for the tangible user interface and advising software tools to facilitate object detection and interactive system development.

Prototyped

This module is involved from the initial phase of methodology, since from the beginning of the development of a system it is necessary to test parts with a multitude of objectives to: Verify functionalities, find out aspects related to the system interface, validate navigation, try new possibilities of techniques, among others.

Evaluation

This module is involved from the initial phase of the methodology, its objective is to test something. So much to know if it works correctly or not, if it meets expectations or not, or simply to know how a certain tool works. Evaluation is a key point for obtaining usable and accessible interactive systems. In this phase, techniques necessary to receive feedback from users are applied. It also relates to usability metrics and evaluation methods.

4.2 Development Process

The objectives of each development phase are presented below, together with a table specifying the activities that were added for the development of interactive systems based on tangible interfaces and gamification and the products that are generated.

4.3 Requirements

Communication with users is a priority aspect for companies that develop software systems. During this stage it is important to make clear the scope and tool of gamification that the system will have based on the user’s needs.

The activities defined in this stage and the products generated are presented below (Table 1):

Table 1. Requirements stage activities
Full size table

4.4 Design

In this stage the best possible solution is designed, considering that the problem was clearly defined in the requirements stage. Its stages cover different functionalities, activity design, information design, and tangible interface design, as well as the main activities that make up the global interactive systems process.

The activities defined in this stage and the products generated are presented below:

*Note: The evaluations of tangible user interfaces and gamification will be carried out within the evaluations of the design stage within the analysis of evaluation results (Tables 2, 3 and 4).

Table 2. Activities of the software engineering design stage
Full size table
Table 3. Design stage activities for the use of gamification techniques
Full size table
Table 4. Design stage activities for the use of tangible user interfaces
Full size table

4.5 Implementation

Also known as the coding stage, since it is where the necessary software code must be written that will make it possible for the finally implemented system to comply with the specifications established in the requirements analysis stage and respond to the system design (Table 5).

Table 5. Implementation stage activities
Full size table

The activities defined in this stage and the products generated are presented below:

4.6 Launch

In this stage it must be verified that the acceptability of the system has been achieved, through a correct combination of social and practical acceptability. In this phase it is important to have user feedback through tests.

The activities defined in this stage and the products generated are presented below (Table 6):

Table 6. Launch stage activities
Full size table

5 Study Case

Due to the limitations to work with blind students it was decided to make a first interaction of the proposed methodology based on the work of Jafri et al. [24, 25] which presents a tangible user interface application for teaching shape recognition and spatial concepts to visually impaired children utilizing a computer vision based tangible tracking system which employs activities in the form of a game to teach these concepts. They also report interesting suggestions made by experts (i.e., teachers for visually impaired children who evaluated their system) such as providing an audio description of the shape instead of simply naming it, that we applied in our prototype.

For the presentation of the application we replicate the model shown by [24, 25] who divide their system into three components: a tangible tracking system, geometric objects which are used as the tangible objectives, and an application. The details of these components are as follows:

5.1 Tangible Tracking System

To detect and track objects we use ReacTIVision [9] which is an open-source cross-platform computer vision framework for fast and robust tracking of fiducial markers (Fig. 2) attached to physical objects, as well as for tracking multi-touch fingers It was designed primarily as a toolkit for the rapid development of tangible user interfaces (TUI) based on interactive multi-touch tables and surfaces [24, 25]. This framework has been developed by Martin Kaltenbrunner and Ross Bencina as the underlying sensor component of the Reactable, a tangible modular synthesizer that has set the standards for tangible multitouch applications (Fig. 3).

Fig. 2.
figure 2

Stages that make up the proposed methodology.

Full size image
Fig. 3.
figure 3

Fiducial markers [9].

Full size image

There are several ways to configure the hardware for the tracking system [10]. We have adopted the configuration suggested by the METUIGA (Fig. 4) [24, 25] that consists of a 40 cm × 30 cm acrylic, a 5-v lamp and to detect the objects we mount a high definition camera brand Microsoft.

Fig. 4.
figure 4

Inside of our tangible system.

Full size image

5.2 Geometric Objects

Since the interaction with the system is through tangible objects, representative figures of the different basic geometric shapes to be used in our system were developed (Triangle, Rectangle, Square, Circle, Pentagon, Hexagon, Rhombus, Oval, Rectangle), which were reviewed and approved by a visually impaired student. Finally, in each one of them a fiducial marker was added to be identified by our system; adopting Jafri et al.’s approach, the name of the shape in Braille is also added [24, 25].

5.3 Application Developed

An application has been built replicating proposed features by [24, 25] which it is connected to a tangible interface and a tracking system. It has been developed with the objective of allowing the child to learn about geometric objects (Triangle, Rectangle, Square, Circle, Pentagon, Hexagon, Rhombus, Oval, Rectangle) which are available in the application and are identified by means of their corresponding marker.

The child can interact with the application using the tangible interface and receiv-ing feedback through the ear adopting the suggestions made by the experts in [24, 25]. He has two options, in the first one he can put the objects in the acrylic and he will receive feedback of the object that he is putting, in the second option he will be asked questions in the form of riddles so that he responds by putting the figure that he considers is the one we are referring, subsequently the application analyzes if your answer was correct and incorrect informing through audio, the user has three oppor-tunities to respond correctly simulating the lives of a gamma system, when your an-swer the correct one is awarded a trophy based on its performance as well as a possible medal if it fulfills some specific objective (For example: It uses all the tangible objects).

In comparison with the works presented by [24, 25] who present activities for teaching geometry and spatial themes (teaching tactual shape perception, teaching orientation concepts, teaching spatial relationship concepts) we focus on presenting a different way of identifying geometric objects (Teaching tactual shape perception) through riddles, complementing the user experience with the inclusion of mechanics and dynamics of gamification such as the inclusion of levels according to their performance, the delivery of rewards as they achieve objectives, a scoring system, lives, time and progress.

The following are the questions that the child must answer in case he decides to use the play option.

  • I have 6 sides, all the same. Who I am?

  • I have 4 sides, 2 short and 2 longer. Who I am?

  • I have 3 sides, all are equal. Who I am?

  • I have 3 sides, all are equal. Who I am?

  • I have 6 different sides. Who I am?

  • I am fat and round. Who I am?

  • I am long as a building. Who I am?

The following shows the progress made in the construction of the software and the tangible interface following the METUIGA methodology (Fig. 5, Fig. 6).

Fig. 5.
figure 5

Implementation of gamification techniques.

Full size image
Fig. 6.
figure 6

Application built following the METUIGA methodology using tangible user interfaces and gamification techniques.

Full size image

6 Conclusions and Future Work

This methodology is a work that is still in process which aims to support the construction of interactive systems using tangible user interfaces and gamification techniques. Modifications were made at the different stages of the user-centered design, to include aspects related to gamification and tangible user interfaces, since it is a methodology specially designed to support the process of building systems that use these techniques.

A functional prototype focused on our case study was carried out, which shows how the first three stages of the proposed methodology are carried out. The prototype is made until the implementation stage. At this stage the necessary software code must already be written that will make it possible for the finally implemented system to comply with the specifications established in the requirements analysis phase and respond to the system design. These documents establish the idea, the vision of the product, the type of gamification and the scope of the project, an initial functional requirements design, an initial dynamic design and a prototype that could sometimes be considered ‘disposable’, since from this prototype the changes begin in the next versions.

As a future work, we intend to continue with the experimentation of the proposed methodology by creating new applications aimed at people with visual problems, in addition, we intend to extend the functionalities of the presented prototype in order to add more activities that enhance geometric mathematical skills in blind users or with low vision, which will aim to help improve the understanding of figures and bodies which will facilitate the curricular integration of students with special educational needs derived from visual impairment in the classroom. After these improvements, we plan to test the usability of the system with children in local public institutions in the state of Aguascalientes, Mexico to identify any usability problem.