Keywords

1 Introduction

Biofeedback games are games, in which players use their bio-signals to alter dynamics in the game. Often these games are designed to heighten self-awareness of players’ combined mental and physical states, and hence lend themselves nicely for psychotherapy.

Current products address various aspects of mental health, from improving concentration to overcoming anxiety. Thus far, games promoting concentration use brainwaves to manipulate game dynamics. Star Wars’ the Force Trainer has the player concentrate to levitating a ball and lifts the ball depending on differences in several bands of brainwaves [1]. Mattel’s MindFlex is similar, using brainwaves to navigate a ball through an obstacle course [2]. In the realm of gaming, NASA and SmartBrain Technologies have created a game controller that responds to neurofeedback [3]. Players are taught to concentrate more, as the controller becomes harder to move when the player becomes less focused (measured as more slower brain waves).

Another application of biofeedback games is as a way to overcome anxiety. Emanuel Andel’s Knife.Hand.Chop.Bot puts the player’s fears to the test [4]. The player puts his/her hand on a metal surface with fingers spread. Then a robot moves a sharp knife in between the fingers with the exact stub location determined by the galvanic skin response of the player. A less intense game to tackle angst is HeartBeat, a hide-and-seek-like game for children [5]. Each child has a sensor worn around the chest. If his/her heart rate goes above a threshold, a radio signal is transmitted to opposing teammates and the hiding spot becomes revealed. Our project shares this element of fear, in that the game becomes harder to play when players are anxious. Like Knife.Hand.Chop.Bot, our project uses the player against him/herself, though in subdued manner.

Rather than emphasizing angst, some games promote the converse – calmness. Mindball pits two players in a tug-of-war-like game to see who is more relaxed [6]. Journey to Wild Divine, a biofeedback computer game, fosters calmness in its general goal to train mediation skills and self-awareness [7]. Instead of a joystick or mouse, the controller consists of three sensors on the fingers that record heart rate and galvanic skin response. To navigate through the virtual world in a first-person adventure, the player must adjust his/her bio-signals depending on the level. Active Ingredient’s Ere Be Dragons is another first-person adventure game, but for the personal digital assistant (PDA) [8]. The player sets a target heart rate, and the virtual world fades in and out depending of how off he/her heart rate is from the target. Again, it emphasizes self-awareness and trains users to adjust their calmness.

Our project has overlapping goals with its intent on training players to adjust their emotional state and remain calm. But our project requires more: players must dual-task, keep calm while coordinating their movements in response to a changing environment. This additional component potentially makes Heartbeat Jenga most similar to future games for the Nintendo’s Wii Vitality Sensor, a heart rate finger clip extension to the Wiimote [9]. However, to date, the first game still under development that utilizes the sensor Heartbeat

2 Implementation

2.1 Heartbeat Jenga Game

Heartbeat Jenga is a modified game of Jenga. Like Jenga, players take turns removing blocks from a tall tower and placing them at the top. The tower subsequently grows taller and less stable from gaps created by the removed blocks. The game ends when a player causes the tower to fall.

The set-up is also similar to Jenga. It consists of 54 rectangular blocks, placed in groups of three to form individual stories of the 18-story tower. The alignment of each story’s blocks is perpendicular to the alignment of the adjacent stories, so from the side view, you would see the short ends of three blocks in one story and the long end of one block in the story above and the story below. In our version, the tower rests on a vibrating platform that shakes in time with heartbeats. A faster beating heart causes more shakes, whereas a slower beating heart causes less shakes. An additional modification is that the lighting in the room is controlled by the player’s heart rate. A fast rate dims the lights, ultimately turning them off, while a slow rate brightens the lights.

2.2 Scenario Demonstration

The game is played in a dark room and just relied on ambient lighting. At the beginning of a player’s turn in Heartbeat Jenga, the player puts on a bio-sensing necklace device (see Fig. 1-A). Securing the necklace around the neck initiates the player’s turn, which is visualized by the surface surround the tower illuminating by optical fabric during the duration of the turn (see Fig. 1-B). The computer subsequently begins collecting and processing the player’s heartbeat signal. Each time a heartbeat is detected, an ornament on the player’s necklace lights up, and the vibrating platform under the tower of blocks shakes a random amount. The lighting also changes subtly over time to match the fluctuations of the player’s heart rate. For example, when user’s heartbeat rate goes higher, it means he/she is getting nervous. In the mean time, he will lose lighting and the desk will shake more (see Fig. 1-C). While these changes are occurring, the player is removing a block from the tower and placing it at the top. If he/she successfully does so, the illuminated surface around the tower turns off. The vibrating platform ceases to shake, the ambient lighting returns to its brightest setting, and it becomes the next player’s turn (see Fig. 1-D).

Fig. 1.
figure 1

Scenario demonstration: (A) At the beginning of the Jenga game, user puts on a bio-sensing necklace device; (B) The optical fabric is illuminated during the duration of the turn; (C) If heart rate goes higher, the platform shakes more, and the lighting dims; (D) When user takes off the necklace, his/her turn ends.

Full size image

Our necklace device is worn by the player, whose turn it is, and serves as the on-off switch for the vibrating platform and ambient light. It is also soft and flexible, from the use of conductive thread rather than wire for the circuitry (see Fig. 1-A).

When a player starts his/her turn, he/she puts the necklace on. The magnetic clasp at the back clamps down on a force-sensing resistor (FSR), initiating the player’s turn. The top of the vibrating platform lights up as well whenever the necklace is worn.

Next the player sticks the two extending branches of the necklace on his/her chest. Our necklace device measures electrical activity from the heart in the form of electrocardiography (ECG or EKG), and the current prototype uses surface electrodes (OTCTM Reusable TENS electrodes, Koalaty Products, Inc.). One electrode goes by the heart, the other one further away, i.e. over the other breast. (The exact location does not matter, because this electrode is the reference point and carries no pulse signal.)

Lastly, the player connects the ribbon connector from the necklace to the ribbon cable that connects to the Arduino. The ECG signal exits the necklace via this ribbon cable and enters a chip that amplifies and filters the signal (Small 2-lead ECG analog front-end breadboard, Open ECG Project). The processed signal is then fed into Arduino for further processing to isolate the actual heartbeat signal.

Each time a beat is detected, the LED ornament on the necklace lights up. This provides other players with a sense of how calm or stressed the current player is.

2.3 Devices

The devices include heartbeat signal sensor, LED ambient lighting, servo motor, optical fabric and necklace (see Fig. 2).

Fig. 2.
figure 2

Devices

Full size image

2.4 Heartbeat Jenga Installation

Our set-up consisted of three components: a small table with a vibrating platform in the center, circuit boards hidden beneath, and a light above the table to create the ambient lighting. We set an Arduino board and a heartbeat circuit as media to transfer signal from physical input (FSR and bio-signal) to digital output (including LED necklace, vibrating platform by using servo motor, and control of ambient lighting). After getting the bio-signal from the heartbeat circuit, the computer then sends the command to Arduino to adjust the ambient lighting and the shaking rate of the platform (see Fig. 3).

Fig. 3.
figure 3

System framework

Full size image

To make the platform on the table shake, we found a tray with wheels. We constrain its movement by placing a ring around it. To create the shaking movement, we attached a servo motor underneath and added extensions to the motor that push the platform upward against the tabletop to create a shake (see Fig. 4).

Fig. 4.
figure 4

Platform: in order to create vibration platform, we used a tray with wheels and attached the servo motor underneath.

Full size image

The platform illuminates whenever a player is taking his/her turn. This is created from covering on top of the platform with optical fabric (Luminex®, Roman Illumination) that turns on whenever the bio-sensing necklace device is worn.

For our prototype of the heartbeat-controlled ambient lighting, we attached an array of LEDs over our table by an arched wire structure. The brightness of the LEDs followed the player’s heart rate with faster heart rate dimming the lights, and a slower heart rate brightening the lights (see Fig. 5).

Fig. 5.
figure 5

The relationship between heart rate, the ambient lighting, and shake rate of the platform

Full size image

3 Evaluation

Evaluation of the game was conducted during the exhibition days for the Tangible User Interface course. To better control the lighting, the game was set-up inside a dark (dusty) tent. We made but one necklace device, so player experience was limited to one of us. She was questioned. Additional feedback was obtained from visitors in conversation.

4 Results

Player experience in the current set-up is not particularly comfortable. Because we needed to control for light, the set-up had to be condensed within a tent, and sitting on the floor is not as comfortable as sitting on chairs. Further, the player’s movements were limited, as everything was wired. In contrast, the necklace was comfortable and felt like a scarf.

In terms of technical issues, we need more work interfacing wires to fabrics. The reoccurring problem is that conductive thread looses its tension. A possible fix is covering seams with bias tape as recommended by Ali Crockett at Aniomagic [Boyd, personal communication]. More advance signal processing will also cut down on false pulses.

The benefits of the biofeedback game began early on. Improvements in coordination were observed within twenty minutes of demonstrating the game to visitors. In contrast, no noticeable ability to control heart rate was observed during the 3-4 h of demonstration. More playtime may be needed.

From conversing with visitors, it seemed like the game was straightforward to follow. Most people noticed the changes we made linked to heartbeat. The shaking platform was the easiest to pick-up on, then the blinking LED on necklace, and lastly the changes in ambient light. Everyone understood the added difficulty of our game that punished players for stressing out. One visitor liked the potential application of Heartbeat Jenga to teach young kids to control their emotions. We also received suggestions for improving and augmenting our game. One recommendation was to have each player wear his/her own necklace and use a token to denote whose turn it is [Goodman, personal communication]. An interesting extension to the game to make it more social was to have all players’ heartbeats involved in the game. A player could then purposely and suddenly increase his/her heart rate during another player’s turn to make the game harder for an opponent [Ryokai, personal communication].

5 Conclusions

The concept for Heartbeat Jenga was well received. People understood the game and could pick-up on the effects caused by the player’s pulse reasonably quickly. On the technical side, the connecting point between the necklace and wires was a weak point and needs further work. Changing to wearable conductive fabric electrodes will also make future evaluation possible, since the sharing of sticker electrodes lacks hygiene. Additional evaluation from other people wearing the pulse necklace and playing the game is needed.