Keywords

1 Introduction

‘If James Joyce were alive today he’d be working for Google’ [1].

James Joyce’s Ulysses (1922) is often referred to as a hypertext or proto–hypertext because it demonstrates qualities such as non-linearity, interconnectivity, and synchronicity [2]. This is particularly true of the central chapter, ‘The Wandering Rocks’, which follows nineteen different characters as they circulate through the streets of Dublin, each lost in an interior monologue of thoughts and impressions. The characters in ‘The Wandering Rocks’ interact with each other as their paths cross, and they observe and are observed by one another in the social fishbowl that was Dublin a century ago. But they are also seen from above, by a more omniscient form of authorial surveillance. As one scholar describes it: ‘“Wandering Rocks” is a chapter that can be played as a board game on a map of Dublin and that employs surveillance from a vertical perspective as its narrative metaphor’ [3]. Joyce’s painstakingly mapped and detailed central episode, presenting both an omniscient bird’s eye view of Dublin and its inhabitants and a multiplicity of subjective views of the city, in turn makes an apt metaphor for the heavily surveilled and data rich 21st century metropolis.

In a remark to a friend, Joyce is famously quoted as having said: ‘I want to give a picture of Dublin so complete that if the city suddenly disappeared from the earth it could be reconstructed out of my book’ [4]. Drawing on various aspects of Joyce’s Dublin as represented in Ulysses and more recent (and ominous) plans for a digital ‘social credit system’ in China [5], our project encourages participants to consider what 21st century Dublin – the multilayered, hyperconnected, datafied city of the present and near future – would look like if we could hear and see the city as it is constructed from our own data artefacts and traces. But how best to design for ambiguity while balancing aesthetic considerations against complex ideas about big data and surveillance?

‘North Circular’ is a design fiction project and future scenario–based interactive public art installation that invites participants to navigate an urban environment built from big data fragments exposing the political, social, and consumer behaviour of a fictitious populace. The goal of North Circular is not to reconstruct Dublin as it was, but rather the current lives of Dubliners from their own online data presence. We want participants to hear and see, in an abstract manner, the data layer that underlies the city, henceforth known as the datasphere. The emergence of big data can be interpreted in a number of ways: on the positive end of the spectrum, open data can be an enjoyable way of exploring a city and its people; on the negative end, however, data can equate to heavy surveillance, influencing behaviour and interfering in citizens’ private lives. It is this duality that we aim to highlight in our installation.

In this paper, we describe the results of a cultural probe aimed at understanding user thresholds for minimal specification and ambiguity in design in interactive installations. In the initial sections of this paper, we describe our efforts to model the interaction between participant and system, and design prototypes. Through these efforts we explore the limits of minimalism and underspecification in the design of an interactive installation that deals with complex issues surrounding big data and surveillance. This paper asks: How can artists and designers open a meaningful dialogue on the vast complexities of, for example, the datasphere or mass surveillance (not to mention Ulysses) using a minimum of text, image, and sound? Drawing on ideas around productive ambiguity in design [6] and affordances [7, 8], we investigate the tradeoffs between clarity and ambiguity, and functionality vs. aesthetics.

2 Related Work

We position our project as an interactive digital art installation with an analogue heart (‘The Wandering Rocks’) and a particular emphasis on designing for ambiguity, subtlety, and minimalism. On one hand, the ubiquity of data in our 21st-century cities has seen the production of innumerable data-driven art projects that both explore the expressive potential of these new resources and reflect on the rich and challenging experience of life in the digital datasphere [9]. On the other hand, much effort, funding, and research in the digital humanities has sought to preserve, adapt, and reinvigorate analogue works of previous eras to the digital world we now inhabit [10]. As in the case of North Circular, combining these two broad aims has sometimes taken the form of digital projects based on the complex and highly suggestive works of the Irish modernist author James Joyce as they intersect with the topography of Dublin. Three examples of Joycean projects, all of which are based around mapping, include: JoyceWalksFootnote 1; the Walking Ulysses project at Boston CollegeFootnote 2; and Dislocating Ulysses at the University of VictoriaFootnote 3. All of these examples use geospatial data to help users explore the densely multilayered city.

Interactive installations and displays that address themes of big data and surveillance are becoming increasingly common in the art world. This is evident for example in the Big Bang Data exhibition (2016) at Somerset House in London, which gathers together a diverse array of projectsFootnote 4 exploring aspects of the boom in publicly available data. These projects invite visitors to reflect on how we generate and contribute to the growing mass of data, not only actively through social media but passively through our always on mobile devices. Using cutting-edge techniques in data sifting and visualization, the exhibition explores the quantification of individual lives and whole populations, changing forms of communication, ideas of privacy and piracy, consumption and commodification, and so on - often promoting interaction through the unsettling use of visitors’ own data.

In terms of engaging visitors, designers of interactive installations and displays often cite the importance of ambiguity, such as that described in [6], as a means of arousing intrigue, mystery and delight in onlookers [e.g. 11]. Other designs rely on more explicit indicators for drawing in visitors, arguing the need for clear, unambiguous affordances that people can easily identify and relate to [12]. One recent example of an installation that makes productive use of ambiguity and a minimalist aesthetic to engage users with themes around big data, surveillance, and hidden systems is Familiars (2015)Footnote 5 by Georgina Voss and Wesley Goatley. Using intercepted communication signals, the installation maps the trajectories of cargo vessels as they move across the globe by land, sea, and air. Visuals are kept to a minimum of suggestive white tracking numbers and red pathways; the audio soundtrack consists of a cacophony of overlapping radio navigation communications. As in North Circular, Familiars aims to transform the immaterial world of raw data into something material, while leaving room for reflection on the part of users about the potential impact this data and these unseen systems have on their daily lives.

With these considerations in mind, we recognize that there is room for deeper investigation. The tension between ambiguity and more overt physical affordances, for example, remains a relatively unexplored area of research. Through this paper we hope to bring greater attention to the use of ambiguity in the context of ambient displays. We also hope to go beyond the more conventional digital humanities approaches to exploring Joyce’s works. Projects that simply map Joyce onto the topography of Dublin are interesting but limited; they miss the playful, challenging, and capricious spirit of the author, who resisted easy formulas and narrative enclosures. Mobile applications such as Walking Ulysses do not go much further than attaching key passages from Joyce’s writing, along with corresponding historical data, to points on a Google Map. While enhancing an existing map in this way was groundbreaking a decade ago, such projects now appear reductive and literal in their treatment of Joyce’s vision. We instead aim to use Joyce’s vision of the modern metropolis (circa 1904) - heavily surveilled, data rich, a jumble of interactions and crossed paths - as a jumping off point to explore the 21st century urban space in all its post-digital complexity. ‘Post-digital’ in this context suggests a point beyond the digital revolution, where immersion in the datasphere is taken for granted and the interaction and overlap between human and digital worlds becomes a primary focus of artistic investigation [13].

3 The North Circular Project

The North Circular project employs metaphors from Joyce’s Ulysses to look at surveillance but more specifically at voyeurism, creating a space in which the participant is both the observer and the observed. Nearly everyone who uses social media admits to lurking – the practice of using your account to ‘spy’ on others without actually posting. So-called ‘lurkers’ are a majority faction in interactive situations of all types [14]. Participation inequality, described by Nielsen’s ‘900-9-1 principle’ [15], is a defining characteristic of most online communities. This, in a sense, is what the experience of reading the ‘Wandering Rocks’ episode of Ulysses feels like. The reader is given privileged access to bits and pieces of Dubliners’ everyday lives, silently observing their private thoughts and public behaviour, and witnessing the gap between the two, as they traverse the city.

In the installation, we aim to create an immersive, thought-provoking experience of voyeurism. When the interactant enters, they will get the sense of being in a bustling city center, with ambient sounds of the city filling the room. We are exploring the use of sound as minimally as possible, while still conveying these complex ideas, namely, making one’s interior monologue external. As we browse the internet, we are, in essence, internalizing others’ thoughts. What feelings emerge when instead those fragments are spoken aloud? Is it strange to hear vocalizations of text features such as emojis or hashtags? Whose data are we listening to? We plan to gather a limited set of personal information from the visitors and include some of their own public data in the installation. When they hear their own comments, photo captions, tweets, etc. voiced back to them, does it spark reflection on what they are putting into the data-sphere? Do they question whether the other fragments they hear belong to the other participants in the room? Will they talk to each other about it?

3.1 Cultural Probe

To explore the thresholds for minimal specification, we conducted a cultural probe focusing on interactive art installations in London. Two exhibits in particular provided us with relevant content from which to obtain user feedback, each representing a different end of the spectrum in terms of the affordances they offer to convey specific ideas to an audience.

The first was Empty Lot, an interactive living sculpture commissioned by Hyundai for the Tate Modern museum and designed by Abraham Cruzvillegas. The large-scale piece filled the Turbine Hall; it consisted of wooden scaffolding, similar to that of a ship, holding over 100 boxes of dirt taken from various parks around London. The artist described the living sculpture as a whole as being ‘made out of hope’ (Fig. 1).

Fig. 1.
figure 1

Empty lot exhibit (Cruzvillegas) in the turbine hall of the tate modern museum

Full size image

The exhibition text suggested hidden layers of meaning - social, personal, and cultural - beneath the simple concept of waiting for something to grow. The idea was that with daily watering and sunlight, organic material would start to grow out of whatever was in the dirt. The space was so large, however, that many visitors did not read the instructions and therefore did not realize that they were being encouraged to throw objects into the dirt.

During two hours of observation of visitors’ reactions, the average amount of time each person spent looking at the piece was only three minutes. There was some confusion among visitors who claimed they ‘just don’t get [this] sort of thing’, i.e. putting empty planters in a museum space. Several people we interviewed brought their own background to their interpretation: a landscape architect told us that nothing was likely to germinate in December; another visitor said that it reminded her of what she learned in school about explorers bringing seeds home from the New World.

The second exhibition we observed was States of Mind by Ann Veronica Janssens at the Welcome Collection. There was so much interest surrounding this exhibition that a long queue had formed, with only ten visitors admitted at a time to ensure that each person could experience the room properly. The installation played on individual perception: the room was filled with multi-coloured mist and was intentionally disorienting. Visitors could see only one meter or so ahead. We were interested in the fact that visitors knew exactly what to expect based on widespread publicity, but they came anyway. In the queue, iPads were given out that provided demonstrations of famous perception experiments, but these visuals sparked only minimal conversation. Visitors tended to stick together while inside, which somewhat undermined the goal of challenging individual perception of time and space. Visitors seemed to treat the exhibition almost like an amusement park, complete with rules and regulations to guide enjoyment.

3.2 Defining the Concept

Our cultural probe findings led us to strive for an adequate balance between minimal specification and ambiguity on one hand, and clear affordances and specificity on the other. We started with a vision for North Circular (Prototype I) in which we attempted to synthesize a number of divergent strands on the main themes of Joyce, big data, and surveillance. We later devised a simpler, more site-specific model (Prototype II) wherein we sought to avoid overburdening the visitor with too many stimuli. We also tried to avoid some of the pitfalls observed in the States of Mind exhibit (see Sect. 3.1) by ensuring that the experience left space for exploration and discovery.

In the initial prototype we conceived of two separate rooms, one for the observer (‘outside’) and one for the observed (‘inside’). The observed participant was immersed in the subjective experience of the digital datasphere, while the ‘outside’ observer enjoyed a more omniscient, godlike view of the ‘inside’ room. In the second prototype, we refined the concept to require a single room - a long corridor with elevators at both ends for descending to and ascending from the space - in order to combine the passive experience of being ‘spied on’ with the active voyeurism of observing and consuming others’ data traces. Both prototypes are further elaborated below.

Prototype I. Our initial concept invited participants to assume a surveillance role in which they observed the principal interaction taking place inside the exhibit’s central space. The first stop was a surveillance area, which gave users a chance to understand what was happening inside; the second stop was a larger main room for exploring and interacting with the visually minimalist and binaural urban social space.

The main room was dimly lit and featured brightly coloured floor projections and multiple soundtracks played through a wireless headset. Circles containing basic personal data (age, sex) and a three–digit citizen score were seen moving around the space. Meanwhile, through kinetic typography, blue lines of objective data (e.g. headline and financial news, transportation check–ins, census figures) suggested paths through the city. Four separate binaural audio loops represented subjective data streams (e.g. a telephone conversation, text–to–speech enabled tweets, private thoughts, missed connections). Every crossed path and data stream encounter appeared to impact the citizen score shown inside the moving circles. The participant could traverse this urban labyrinth and reflect on how their decisions, as well as chance encounters, might affect their own individual status in this heavily surveilled environment (Fig. 2).

Fig. 2.
figure 2

North circular prototype I depicting participants’ surveilled movements, interactions and citizen scores over time

Full size image

Joyce does not represent Dublin visually in Ulysses, argues one critic, but ‘through the minds of the Dubliners we overhear talking to each other’ [3]. We do not see the city directly; we soak it up indirectly, through surveillance. Outside the main room, on the other side of the surveillance partition, a row of screens and headsets could be found, enabling the participant to listen to the conversations and other personal material on the audio channels being played inside and watch a live video feed of the interactions occurring in the main space, as the names and scores of fictitious citizens scrolled rapidly down the right-hand panel.

Prototype II. We refined our concept with a particular testing environment in mind, i.e. the corridor that runs through our institute. The space needed to be confined enough to draw visitors past the simple, motion-activated speakers, but large enough to encourage circulation. The space we chose had elevators at either end of the hallway, thus allowing for a staging area upstairs where we could take time to scrape the internet for visitors’ public web data. The elevator itself served as a transitional space with an information card describing North Circular. The series of rooms opening onto the corridor held projectors, which were placed to display abstract visualizations of data on the walls (Fig. 3). This was intended to give the visitor a preliminary sense of what the exhibit was about, i.e., comprehending and reflecting on all the 1s and 0 s we create in our everyday lives.

Fig. 3.
figure 3

North circular prototype II depicting an aerial view of the installation space

Full size image

One question that came up in the design of this prototype was: Which types of data are interesting to listen to? The point was not to overwhelm the user with the enormity of all the data out there, but rather give them a chance to reflect on how decisions and chance encounters might affect their data, personal interactions, etc. Using data sources such as Craigslist’s ‘missed connections’ would lead the user to reflect on the ways people try to connect in the modern age. On the technical side, motion sensors were used to activate Irish accented text-to-speech fragments of real Dubliners’ online data. These sensors were intended to encourage circulation (an important Joycean theme) around the space to simulate exploration of the datasphere.

We also experimented with ways of gathering visitor data. For testing purposes, it had to be flexible enough to perform a Wizard of Oz experiment using people’s names and postal codes or other commonly accessible information. Once we knew what kind of information was easily and quickly available, we would be able to devise a way to automate the process. The correct ratio of personal data to random data fragments also had to be achieved. If there were 19 snippets (reflecting the 19 characters in ‘The Wandering Rocks’), we hypothesized that four should be enough to be certain a user would hear something from his or her own life and therefore be encouraged to further explore the space. The data snippets should also be banal enough that only the interactant knows who created the comment, thus avoiding violation of privacy by revealing actual addresses, phone numbers, etc.

4 Discussion

At this stage in our project we believe that the union of Ulysses with themes of big data and surveillance is one worth pursuing. Yet these rich and complex themes require significant unpacking. Without proper handling there is a danger that the weight of these themes could potentially overwhelm the visitor and make for a poor interactive experience, obscuring rather than illuminating the topics at hand. So the question arises: how to winnow the content to a manageable, comprehensible package? In our attempts thus far we have tried to avoid overwhelming users with the complexities of Joyce’s text directly, instead extracting from Ulysses the ideas around big data and surveillance that are still relevant today. While interpreting Joyce for the 21st century was one element of this project, we also hope that it will be a two-way street: helping people to better understand Joyce’s most famous work, and using Joyce to help us understand the complex daily interactions of our 21st century cities.

In our cultural probe, we witnessed two very different approaches to the design of interactive installations. Both were striking in their minimalist aesthetic, but each had its own pitfalls. The first, Empty Lot, left too much unsaid, creating confusion among visitors and discouraging sustained engagement. The second, States of Mind, while apparently more successful in conveying the meaning of the installation, failed to promote deep reflection, achieving only a superficial level of engagement more akin to entertainment. With these experiences in mind, we sought to design an installation that produces an immediate impact on the visitor, connecting him or her with the concepts being presented, while inviting people to draw their own conclusions. We also sought to create a personalized experience by employing users’ own data as part of the installation.

Our prototypes I and II show a progression in the visioning of North Circular from an overly complex tangle of ideas, stimuli, and role playing (observer, observed) to a more simplified, streamlined concept whose spatial characteristics give the installation a clear start and end point (missing from Prototype I, which lacked a defined path). Prototype II also jettisoned as too heavy-handed the motifs of citizen scores and an external surveillance control room, replacing them with a less narrativized, less explicitly dystopian experience. We aimed for an experience that opens up space for reflection on themes of big data and surveillance without prejudicing the user’s response. We achieved this by: focusing the visitor’s attention on motion sensor activated audio while reducing visual cues; inviting interactants to take notice of their contribution to the data layer by uncovering minor data fragments that were forgotten or even unknown to the user; and relying on wall displays only to afford a general contextual understanding of data immersion. The decision to use visitors’ own data fragments to personalize the engagement was inspired by Empty Lot and the possibility of allowing visitors to bring their own seeds to plant. We posited that interactants would be met with surprise when they heard their contributions to the datasphere as synthesized utterance fragments, and this would provoke a shock of recognition and a deeper level of engagement.

With these decisions in mind, we present five insights and design recommendations for balancing complex themes with minimal specification in the design of interactive installations and displays.

  • Tailor content to the user to make it relevant.

  • Create a clear path through the installation space.

  • Encourage active engagement and reflection through a combination of playful ambiguity and affordances that interactants can recognize and relate to their own experiences.

  • Don’t inhibit reflection by telling interactants what to think. Instead, allow interactants to muse freely on complex themes.

  • Create a clear signal by reducing noise from competing modalities.

5 Conclusions and Future Work

Returning to the quotation at the start of this paper, Tom McCarthy wrote of Ulysses that Joyce’s ambition was ‘to make a whole culture, at micro- and macro-level, from its advertising slogans or the small talk in bars to its funerary rituals and the way the entire past and future are imagined’. In other words, constructing Ulysses was a bidirectional process: in one direction, building a text, city, and culture out of data fragments; in the other, making it possible to reconstruct that city and culture from the text (should anything happen to ‘dear dirty Dublin’).

This was our initial inspiration for North Circular, which seeks to apply the insights and provocations of Joyce’s text to similar but evolving concerns raised by big data and surveillance in the data rich 21st century city. When we looked at previous attempts to use digital technology to in some sense ‘update’ Joyce, we found them to be overly literal in a way that goes against the playful, experimental, endlessly inventive nature of Joyce’s writing. Using digital technology to update analogue texts without exploring contemporary parallels - Joyce’s Dublin to the digital datasphere - is a missed opportunity to ‘make it new’, in the words of fellow modernist Ezra Pound.

On the issue of ambiguity, we observed in our cultural probe the fine line between the kind of productive ambiguity posited by Gaver et al. [6] and the baffling ambiguity that causes visitors to turn away from art installations. Once again, Joyce’s novels - not only Ulysses but also Finnegans Wake (1939) - show us the way forward, the means of achieving an appropriate balance. Joyce uses rhetorical devices such as omission and paronomasia to add layers of ambiguity to his otherwise highly structured texts. (In Finnegans Wake, for example, a typical instance of wordplay is: ‘they were yung and easily freudened’ [FW 115.21-23].) In this way Joyce’s writing simultaneously reflects the intricate systems underpinning nature and human society, and the unpredictable, serendipitous, infinite possibilities of everyday life.

Finally, our future work involves testing the current prototype and experimenting with different types of data fragments. The goal will be to assess which data sources interactants find to be most compelling and thought provoking, and to determine how well our curation of audio and visual elements enhances the experience. We will also explore interactants’ privacy and security concerns in the use of their personal data fragments.