Keywords

1 Introduction

Development is usually considered as a process of growth and maturation that an individual undergoes throughout the life span. Children are naturally inclined to create play situations and explore their environments. Through play they learn, practice and improve skills, involve in social roles and experience emotions. Play is widely used in therapy to treat children’s emotional and behavioral problems because of its responsiveness to their unique and varied developmental needs [6].

Nowadays interactive technologies provide the means to achieve a radical transformation of play much beyond desktop computers games. According to [4], a large number of products that incorporate interactivity is available to young children, such as musical keyboards, programmable and radio-controlled toys, etc. This range of toys and devices is part of a move towards pervasive or ubiquitous computing in which technology blends into the environment and is not necessarily visible. Ambient Intelligence (AmI) refers to electronic environments that are sensitive and responsive to the presence of people. According to [1], the AmI paradigm builds upon pervasive computing, ubiquitous computing, profiling, context awareness, and human-centric computer interaction design. AmI environments offer opportunities for supporting the playing needs of children and examine a variety of ways in which ICT can be integrated into playing situations.

Designing and creating playing experiences under the perspective of AmI has the potential to provide enhanced gaming experience to users and in particular to children. Such games are facilitated by systems and technologies that: (a) are embedded in the environment, (b) can recognize children and their situational context, (c) are personalized to their needs, (d) are adaptive in response to young children interaction and (e) are anticipatory to children’s desires without conscious mediation.

2 Background and Related Work

2.1 Augmented Artifacts for Play and Play Environments

Embedding interactivity into physical objects allows supporting traditional exploratory play with physical objects that can be extended and enhanced by the interactive power of digital technology [168]. A literature review revealed various smart artifacts targeted to support young children’s play through enriched interaction and enhance their learning skills. For instance, I-Blocks are physical artifacts that support not only learning by construction, but also programming by building [12]. ‘Cube to learn’ is a tangible user interface for the design of a learning appliance [20]. It provides a general learning platform that supports test-based quizzes where questions and answers can be text or image based, employed in a common shape cube. Smart Puzzle, a two-dimensional puzzle, enhances the ability of students to solve mathematical and scientific problems [19]. Similarly, the Smart Jigsaw Puzzle Assistant [5] uses miniature RFID tags and a RFID scanner in order to provide to each puzzle piece its unique ID. Another approach to augmented artifacts in children entertainment and education is the Farm Game, a tangible tabletop application for children [13]. The main objective of the Farm Game is to contribute enhancing both motor-skills and cognitive development. FaTe [8] allows young children to play, communicate, explore, build their own stories and create their own narrative flows in a collaborative environment. Similarly, Kidpad [10] is a collaborative storytelling tool for children, which allows them to create their own hyperlinked story scenes and link them together in a two dimensional space. Along the same lines, Story Toy [7] lets the children play with linear and branched narrations on a toy farm with electronic sensors making use of voice but not of images, whereas SIDE Project [14] aims to improve the social skills of adolescents with Asperger’s Syndrome.

2.2 Tangible User Interfaces for Play

Tangible interfaces have the potential to provide innovative ways for children to play and learn through novel forms of interaction [15]. Example of such a tangible interface is the I/O Brush, where children play using special paintbrushes which they can sweep over the picture of Peter Rabbit in the classic storybook [17]. Chromarium is a mixed reality activity space that uses tangibles to help children aged 5–7 years experiment and learn about color mixing [16]. The Telltale system is a technology enhanced language toy which aims to aid children in literacy development [2]. It consists of a caterpillar-like structure and children can record a short audio clip in each segment of the body and can rearrange each segment to alter the story.

2.3 Occupational Therapy and Technology

Occupational therapy (OT) is a client-centered health profession concerned with promoting health and well-being through occupation. Play is one of the areas of human occupation that OT focuses on, and appropriate activities for children are widely used in order to evaluate and facilitate the development of their skills and abilities. Technological advancements continually influence occupational therapy practice methods and create new tools for intervention. Nowadays, there is a large range of software solutions that promote the development of a child and monitor potential developmental issues, as well [3, 11, 21].

3 Technological Infrastructure

In the context of Ambient Intelligence, a set of augmented artifacts was developed to extend the “means” that the child can use during playing such as: (a) force-pressure sensitive table surface, (b) augmented pen, and (c) digital dice. These artifacts supplement existing technological infrastructure including an augmented interactive table called Beantable [22] and a cross-platform remotely-controlled three-dimensional avatar called Max [23]. Beantable supports preschool children development and it is made up of technological components that offer to the child the opportunity to engage in virtual (–based) play situations either alone or with the presence of the Max virtual partner. Max can act as a guide, assistant or information presenter for novel, cross-platform Ambient Intelligence (AmI) edutainment scenarios. The role of Max depends on the client-application’s requirements. In order to achieve natural communication channels, both non-verbal and verbal behavior are essential. Non-verbal communication includes full body animation and facial expressions.

3.1 Force Pressure Sensitive Interactive Surface

A force pressure sensitive interactive surface can be used as an alternative medium for communication and multimodal interaction. Firstly, it can act as an alternative joystick, and secondly, it can be used to identify the psychological and physiological state of the children through monitoring somatic and behavioural components. Beantable has a horizontally vision based-back projection interactive surface. This setup allows the installation of four force pressure sensitive resistors beneath it, as shown in Fig. 1(A, B). Each sensor varies its resistance depending on how much pressure is being applied to the sensing area. Despite the fact that these sensors are not extremely accurate, a software module uses them as a scale to capture information about the pressure exerted by the user during interactions. Every single sensor is attached to a FlexiForce adapterFootnote 1 which provides analog input to the Phidget Interface Kit,Footnote 2 as shown in Fig. 1(C).

Fig. 1.
figure 1

Deployment of force resistors to recognize pressure during interaction

Full size image

The software module receives the analog values from the Phidget Interface Kit (connected via usb to the pc) and calculates the two-axis position of the exerted pressure using torque and rotational motion equations. Additionally, it measures the total force in pounds (lbs) by summing the applied force that each sensor receives. Because of the weight of the translucent Beantable’s surface, as well as the sensor’s inability to return always to the initial condition (as was before user’s touching), the software module runs internally an auto-reset algorithm for calibration purposes. The latter ensures that whenever the interactive surface is empty (no touches, no physical objects, etc.), the captured data are used as a starting point for the next user interaction.

3.2 Augmented Pen

Children’s drawing is a way to improve physical, social, emotional, cognitive development, and at the same time a way to have a lot of fun. The manipulation of a pencil reveals numerous indications of the maturity level of children’s writing skills. As a result, the augmentation of a pen is considered very important in order to measure the applied pressure, the position and the orientation of the pen on the writing surface. The augmented pen allows developers to create new and innovative pen-centric user interfaces and learn how users are affected by them. It is a custom made pen in which all necessary hardware micro-electronic parts fit together while the measured dimensions do not exceed \( 2 {\text{cm }}\left( {\text{L}} \right) \times 2 {\text{cm }}\left( {\text{W}} \right) \times 16 {\text{cm }}\left( {\text{H}} \right) \). The augmented pen (see Fig. 2) was designed with SolidWorks and came to reality via a 3D printer.

Fig. 2.
figure 2

Augmented pen assembly

Full size image

As Fig. 2(A) depicts, various micro-electronic parts are used for measuring the applied pressure weight, the position and orientation on the screen, and the movement acceleration. In details, the pen consists of: (a) an infrared led which lights automatically when the stylus is pressed on its tip against any surface, (b) a force sensitive resistor in a specific embodiment in order to directly receive the pressure exerted by the user during writing, (c) a small, thin, and low power triple axis accelerometer, (d) a small microcontroller (i.e., Arduino Pro Mini), and (e) an RF link transmitter at 315 MHz. Embedding the aforementioned hardware infrastructure into the pen does not impact negatively its weight, which remains under 20 gr.

The augmented pen uses no ink, but infrared light to make drawings. Due to the use of the infrared led, the augmented pen is suitable for interactive surfaces monitored by infrared spectrum cameras as in the case of Beantable. The force sensitive resistor is attached to a custom made adapter which provides analog input to the microcontroller. The latter gathers input from every sensor and transmits them as raw data wirelessly to the RF receiver attached to another microcontroller connected to the computer. Moreover, an appropriate software module runs in the background in order to receive augmented pen’s transmitted raw data. Such module computes the rotation matrix using the accelerometer values as well as the total force in Newton classifying the applied pressure as soft, normal or hard.

The augmented pen is powered with an ultra-small lithium rechargeable battery with a capacity of 40 mAh and by utilizing the microcontroller’s low power consumptions settings it can run continuously for almost a week. Nevertheless, the battery can be easily recharged via a micro usb socket placed beneath the cap (see Fig. 2(D)).

3.3 Digital Dice

Several traditional games such as board games include various types of dice (e.g., with dots, numbers or colored sides). Playing dice games is a good way for children to learn game skills like taking turns, staying on task, mentally adding numbers, observing others’ game, etc. An augmented physical dice allows the system to be able to recognize its motions and identify its top side. This approach allows children to continue using a typical dice in a traditional way, while the results of a roll can be automatically retrieved by the system.

DICE +Footnote 3 is an interactive gaming dice already available on the market (see Fig. 3). The choice of DICE + was made to give children the affordance of using traditional-looking dice while supporting the wireless communication of the rolling results to the registered clients. DICE + provides: (a) an accelerometer with the magnetic field sensor resulting in 3D orientation, (b) 6 independent touch sensitive faces, (c) 6 independent LEDs that can glow in any color, (d) a battery that gives 20 h of game playing, (e) a micro-USB for easy charging, and (f) a thermometer. Due to the fact that DICE + communicates only with mobile devices, an iOS device (i.e., iPad) was selected to host a service application acting as the mediator between the AmI environment and the dice. The implemented iOS service employs a middleware network layer in order to expose its functionality to the network clients.

Fig. 3.
figure 3

DICE+

Full size image

4 Tower Game

The Tower Game has been designed based on OT expertise, so as to meet the needs of OT common practice. Moreover, it has provided the knowledge needed in order to support the monitoring and adaptation logic employed by the game. The Tower Game builds upon sound developmental theories and the definition of expected skills and tools to provide the scientific basis for the rationale of the game [9, 18]. The game is organized into four levels, each targeting a specific age within the range from 3 to 6 years old, following the developmental expectations for child’s play performance in specific age related activities. Each level of the game includes different activities required by young children.

The Tower Game is tailored to the needs of preschool children and supports playing through tangible interaction with augmented artifacts. More specifically, the game allows children to learn, identify and compare six different colors on each side of the dice with those illustrated on the path of the game. Using the aforementioned technological infrastructure this game extends the age range supported by the original game. This is achieved by increasing the difficulty and playing demands according to developmental standards, while providing runtime adaptation based on the child’s level of performance.

4.1 Adaptation Logic

The main adaptation concept employed by the game during play, is that the game is responsible to monitor and evaluate the play performance and commit a representative score to the Adaptation infrastructure mechanism (ADAM). The latter provides the child’s profile, which consists of basic information such as name, surname, birthdate, etc., as well as problems involving functions and structures of the body, activity limitations and participation restrictions. Furthermore, ADAM analyzes the play performance of the child at the current game’s level and makes appropriate adaptation suggestions back to the game. The analysis is conducted using time series methods (i.e., weighted moving average). Using this analysis, the recorded data are imported to the time series in order to generate the developmental curve of the targeted specific activity of the currently active game level. Through statistical analysis, ADAM can not only isolate possible errors and extract the current developmental capacity, but also it makes a prediction about the developmental rate. As a result, the adaptation logic is able to identify children whose development deviates significantly from the one expected at their age. This implies that further investigation is recommended to determine if there are any problems that require treatment. Additionally, using this information, the game can adapt to the child’s evolving skills so as to choose the most appropriate level according to child’s estimated abilities.

4.2 Design

Tower Game has simple rules and offers opportunities to promote knowledge and practice fundamental skills such as color and number recognition, counting and writing. During play the child can acquire knowledge of concepts such as seriation, counting and ordering and at the same time practicing fine hand use and other skills related to numeracy and the use of writing implements. The child also learns to integrate sets of actions so as to follow rules and coordinate movements.

Starting Point of the Game. The game is started through the Beantable’s startup screen by selecting the image of the “Tower”. At the beginning of the game, Max asks the child to place his identity card on the surface (see Fig. 4 Left, Fig. 5 Left). The system recognizes the card and remotely requests the child’s profile from ADAM. The system initializes the game, by using the received profile, and Max welcomes the child with his/her name. At this point, Max asks the child to find and place the “Tower Box” on the Beantable’s surface (see Fig. 5 Right). Thereafter, the game will be started at a level corresponding to the child’s profile, while Max explains by giving relevant instructions. If no action is performed, after waiting a period of time, Max says a “Good bye” message, and the game is terminated.

Fig. 4.
figure 4

Tagged “identity cards” with lanyards (Left). Interactive gaming dice (Right).

Full size image
Fig. 5.
figure 5

Waiting for the identity card (Left). Waiting for the “Tower Box” (Right)

Full size image

Gameplay. Regarding the gameplay, the child rolls the colored dice (see Fig. 4 Right) and moves forward until he reaches the first tile of the same color. In the case of a numbered dice, the child must recognize the number that he rolled and move forward that number of spaces. Two sets of numbers are used, 1–6 and 7–12 according to child’s maturity. If the performed action is correct, he rolls the dice again and continues the previous procedure. If the dice roll is white, a card appears; this card has a random position on the path to which the young child should move. In case he reacts correctly, the card will be added to a staple of similar cards next to the display. The last tile of the path represents the entrance into the tower, however if the player rolls too high, he has to move backwards. The round is over when the rolling result is the exact color or number needed to land in the tower and the player has gathered all the cards. If the child lands at the tower but some cards are missing, Max asks him to continue playing by rolling the dice again and explains him the reason why (i.e., “You haven’t selected all of your cards, throw the dice again”).

At the first game level (from age 3 to 4), the child has to move over the path-maze by touching the tiles one after the other using his finger until he reaches the tile with the same color as the rolling result of the colored dice (see Fig. 6 Left). If the child does not understand Max instructions (i.e., “roll your dice”), the first tile of the path starts blinking followed by an appropriate voice message. If the rolling result is white, the player has the opportunity to save a random positioned animal by moving forwards or backwards to its position.

Fig. 6.
figure 6

An indicative instance of the first level (Left). Player has to drag his finger (Right)

Full size image

While playing the system counts errors, or unsuccessful user performance, for each rolling result. The child’s progress and potential errors are continually reported to ADAM for further analysis. When finishing the game, the adaptation logic is informed and the next two phases are initiated as following:

  • Congratulation phase: Max congratulates the child by saying a random message such as, “Congratulations”, “Bravo”, “Very well”, etc.

  • Free play mode: At the end of the “congratulation phase”, the free play mode starts. During this mode, the child can interact with the system without errors being signaled. The child can freely roll the dice and interact with the game. In this way, the child can create his own play situation and enjoy the game undisturbed. When the child stops interacting for a few seconds, this mode is disabled.

At the second level (from age 4 to 5) the system has similar functionality as in the first. The child has to move over the path by dragging his finger over the tiles one after the other as drawing a line until he reaches the color-name (see Fig. 6 Right). During the game, if the child faces a significant difficulty or makes wrong movements, the system repeats the color name in order to help him move on. Following the same concept as at the first level, when the child drags his finger over the tiles they illuminate for a few seconds.

At the third level (from age 5 to 6) the dice depicts numbers. After rolling the dice, the system reads the number and instructs the child to count as many tiles on the path in order to land on the tile that corresponds to that number. While the child drags his finger over the tiles, they lighten as long as his movements are correct.

Finally, at level four (from age 6 to~7) the functionality of the system remains exactly the same as at the previous levels, apart from the fact that the child has to move over the path by dragging the augmented pen over the tiles as drawing a line in a maze (see Fig. 7).

Fig. 7.
figure 7

The use of the augmented pen over the tiles

Full size image

5 Deployment Setup

The Tower Game has been deployed in vitro within the AmI classroom simulation space of the FORTH-ICS AmI Research Facility as shown in Fig. 8. The deployed technological infrastructure includes Beantable, and the avatar Max both as a playmate running on an iPad mini installed on top of Beantable and as a standalone avatar on a large 55” display. Furthermore, within the AmI simulation space the adaptation logic is deployed to capture interaction data and produce the appropriate adaptation.

Fig. 8.
figure 8

The entire setup

Full size image

6 Evaluation

A preliminary evaluation of the augmented artifacts and the Tower game was conducted involving expert walkthroughs with three accessibility and usability experts from the Human-Computer Interaction Lab of FORTH-ICS. The main objective was to assess the overall system usability and provide recommendations on how to improve the design. The experts were asked to play the developed Tower game and use the developed augmented artifacts in order to uncover any potential violations of usability standards in the design, as well as identify any areas of the design that could potentially cause problems specifically to children.

Overall, the experts found the design of the Tower game intuitive and engaging and pointed out some minor problems. Experts noticed that on the one hand Max was very slow in his reactions and additionally that the game’s idle time was expiring too early without allowing children to think or plan how to play. They also expressed their concern about the size of the secondary screen presenting Max and expresses the opinion that it should be bigger. Furthermore, experts found that the monitoring and adaptation logic of the fourth level should not be as tolerant as it was. They suggested that the voice of the Max should be louder and less computerized. They also suggested that Max should always announce the rolling result after every dice roll. Regarding the physical design of the augmented pen, the experts found it ergonomic, although they recommended that it should be slightly lighter and considerably smaller. Finally, the experts commented very positively both the design of the game and the adaptation mechanism used to dynamically adapt the game to the developmental characteristics of children while playing.

7 Discussion and Future Work

This paper presented the design and implementation of novel augmented artifacts which can be used to enhance children’s play experience, influence occupational therapy practice methods and create new tools for intervention, through popular traditional games implemented in AmI environments. These artifacts are: (a) a force-pressure sensitive interactive surface that can be used to identify the psychological and physiological state of the children and additionally it can be used as an alternative medium for interaction, (b) the augmented pen which measures the applied pressure, the position and the orientation of the pen on the writing surface, and (c) a digital dice that facilitates traditional board games. Through the augmented artifacts, the game became capable of monitoring and following the progress of young players, adapting accordingly and providing important information regarding the abilities and skills of the child and his inferred development progress over time. This was achieved by employing OT knowledge aiming to form the adaptation logic employed by the game. Apart from the developed artifacts, the game involved the use of an interactive table for preschool children and a remotely-controlled three-dimensional avatar. The presented artifacts as well as the introduced game will be evaluated in the future in the context of a small-scale study with children of the aforementioned age group, their parents, and early intervention professionals.