1 Introduction

The game begins with a cut scene of the two player-characters having an argument with one another. Shortly after, it resolves to the ‘Next Day’ where the players take control of the characters and seek to reunite them under the guidance of supporting audio and visual cues. The fractured landscape doubles as a metaphor for the characters’ internal struggle as they navigate a fragmented space filled with obstacles of memories past and other puzzles as they strive to reach one another. The space is composed of interiors and exteriors that reveal and conceal not only the characters, but also obstacles that are meant to be avoided. These obstacles are the remains of the argument that took place ‘the day before’, and need to be navigated around as they patrol the environment. More so, they also reveal in some cases the path that players must take, and should be considered carefully as they make progress. Additionally, lights will turn on as you enter buildings to help guide you, as well as the NPCs which are scattered throughout the environment. In ‘Wall Mounted Level’, knowing the position of the other player (your goal) is just as important as knowing where your own character is. Thus we decided to connect the characters visually with a digital string that also turns red when either one of them touches an obstacle. This visual language parallels the collaboration between the players themselves who are also connected to one another physically.

2 The Environment

The game environment is a hand drawn cityscape that was laser-cut and assembled into a paper relief sculpture. Serving as a metaphor of the characters’ internal struggles, we decided to further fragment the sculpture and create a deeper relief out of it. Other materials that we considered for the construction of ‘Wall Mounted Level’ included cardboard, chipboard, wood, and acrylic plexiglass, but ultimately, we favored the inherent quality of paper as a tangible, vulnerable, interactive medium. These qualities also resonated with the characters’ fragile relationship while inspiring interaction between the players.

The flatness of the paper, however, too closely resembled the 2D screenspace of monitors and flat screens. Wanting to leverage the physical quality of the material, we were compelled to abstract the level into 3D space which resulted in the creation of a relief sculpture. Using a laser cutter, we separated the level into six distinct chunks that we cut and reassembled into a wooden frame to be hung on a wall (Fig. 1).

Fig. 1.
figure 1

‘Wall Mounted Level’ (24” × 36”, laser-cut illustration board and paper). Sitting on an easel in front of projector. Testing dimensions and footprint of entire setup. November, 2017.

Full size image

3 Mechanics

All of the digital elements are projected onto ‘Wall Mounted Level’ using projection mapping and other compositing techniques. As characters move through interior spaces, we mask them by projecting black pixels onto a sorting layer in front of them. At the same time, windows and doors light up in a sorting layer behind them to indicate which building they are currently located in. This helps the players keep track of their own location as they move throughout the level. Roaming NPC’s and enemy AI (the remnants of the argument that took place ‘the day before’) also help to describe the overall flow of the level.

As enemy AI patrol the level and attempt to thwart player progress, ‘good memories’ (the blue orbs) are scattered throughout the level which can be picked up, listened to, and used to revive the characters’ health. Tethering characters to one another also reinforces our narrative of connectedness while prompting deeper cooperation and collaboration between the players themselves (Fig. 2).

Fig. 2.
figure 2

‘Wall Mounted Level’. Players in pursuit of one another while being trailed by enemy AI. Lit interiors help to guide them. Note the blue tether that connects the characters to one another. November, 2017. (Color figure online)

Full size image

4 Technology

Because the level was no longer flat after abstracting it into 3D space, single-exposure projection was not a suitable option given the range of depths. Using the software Isadora [1] allowed us to project multiple exposures at the same time and corner-pin them independently. This provided us the flexibility to further fragment the level while yielding greater agility in designing it.

Unity 3D handles our player collision, game logic and scripting in addition to managing and compositing all visible elements including the characters, their tether, enemy AI, NPC’s, pick-ups, lighting, and black pixels used for masking. Employing two handheld controllers and a Makey Makey [2] for player input, Unity sends its live feed through a Syphon client to Isadora for handling projection mapping.

5 Modes of Engagement

The modes of player interaction parallel the sculpture itself in terms of digital and physical engagement. Digital interactions account for lateral movement as each player steers their own character through the environment using an analog stick. Moving up and down ladders and transitioning between floors requires players to touch one another and complete a ‘Makey Makey’ circuit that they are connected to, which in turn sends the appropriate input message to the software that is listening for it. However, this input message is shared between both players and will affect them equally if they are in valid positions (e.g. at the top of a ladder, or at the base of a stairwell) when the message is sent. Employing both digital and physical interactions in this manner also provides opportunities for symmetrical and asymmetrical collaboration. As a collaborative effort, lateral movement is an asynchronous interaction whereas vertical movement is a synchronous one that affects both players equally and at the same time. As a result, player collaboration intensifies in tandem with this multimodal experience. Similar to the physical quality of the sculpture, the verbal communication and physical coordination that takes place between the players is especially important to us in terms of human-facing interactions as it extrends the narrative of ‘reconciliation’ (Fig. 3).

Fig. 3.
figure 3

Close-up of player interactions. November, 2017.

Full size image

6 Motivation

Our motivation for creating ‘Wall Mounted Level’ was to embrace tangible surfaces as mediums for digital games to exist in, and for the interactions between players to occur in person. Utilizing real, physical surfaces for our environment helps to secure the sense of presence between the players, thereby increasing opportunities for empathy to exist between them. Furthermore, we increase the value of these human-facing interactions by promoting ‘meaningful choice’ through a game that involves some risk and other shared experiences. Throughout its development, ‘reconciliation’ served as the underlying narrative and concept that drove aesthetic choices, gameplay, and the interface not only between the players and the game, but between the players themselves. It was important to us that the game required the physical presence of the players to embolden the collaboration between them. This aided in establishing a deeper connection between the players and the game world itself which is a tangible, physical object. In conclusion, ‘Wall Mounted Level’ is a cooperative multiplayer game that uses mixed-reality to support its narrative, interactions, game-play and aesthetic elements to promote multimodal collaboration between its players.