2003: Contexts of Collaborative Musical Experiences

Abstract

We explore a variety of design criteria applicable to the creation of collaborative interfaces for musical experience. The main factor common to the design of most collaborative interfaces for novices is that musical control is highly restricted, which makes it possible to learn easily and participate in the collective experience. Balancing this trade-off is a key concern for designers, as this happens at the expense of providing an upward path to virtuosity with the interface. We attempt to identify design considerations exemplified by a sampling of recent collaborative devices primarily oriented toward novice interplay. It is our intention to provide a non-technical overview of design issues inherent in configuring multiplayer experiences, particularly for entry-level players.

Appendices

Author Commentary: Musical Contexts of Collaborative Experiences

Tina Blaine and Sidney Fels

Looking back at this paper written in 2003, it is almost comical to read the reference to the growth of the internet facilitating “...a new genre of collaborative interfaces that allow players to communicate over a network from non-specific locations, from virtually anywhere in the world.” Since then, a variety of new collaborative music making experiences have evolved that integrate live coding, real-time composition, wireless audio environments and more. Further, new realms of remote collaboration are enabled by high speed networks, online social networks, smartphones, streaming audio, and increasingly ubiquitous sensor networks. These distributed, networked environments are ripe for designing musical experiences that have the potential to engage an unprecedented number of users. The ability to have commercially available devices with a range of built-in sensors and sound synthesis in the palm of your hand has influenced the development of apps and musical innovations on a grand scale. For example, smartphone app developer Smule claims to have millions³ using their social music network for cloud based jamming and collaborative music making.

One way to frame this explosion of collaborative opportunities is to consider the time-space matrix for groupware (see Table 2 (Baecker et al. 1995)). In 2003, the upper left corner of the matrix dominated the landscape as was clear in our paper. However, examples in the upper right and lower left corners were developing while the lower right corner was nearly non-existent. Today, we are seeing new collaborative contexts that span space and time suggesting that some refinement of our principles are in order.

Table 2 Time/Space matrix for groupware can be used as a frame for considering expanding types of collaborative contexts that can be explored for music making. We include a few of the many examples that have been explored in the corresponding quadrants

In particular, issues related to network latency play a significant role in the Directed Interaction principle. We would consider this a Time-Scale dimension where latencies between 1–30 ms lead to same-time collaboration that feels synchronous. At 30–100 ms, latency begins to be noticeable (Machover et al. 2013), so mechanisms such as external sync or turn-taking become strategies to deal with this delay. Delays of 100–1000 ms inhibit real-time interaction and require quasi-synchronous musical tasks, e.g. such as with Daisyphone (Bryan-Kinns 2004). Finally, delays of minutes, hours and days are purely asynchronous and require network mediation to address the spatial and temporal dislocation of different place and different time-based interactions. Miller (Ramakrishnan et al. 2004) discusses some of these time-scales in conversational contexts for instance.

The internet also enables community building via massive scale opportunities for collaboration, such as City Symphonies (Machover et al. 2013) where urban dwellers contribute crowd-sourced audio materials to compositions that are played by experts. Designing parameters for remote musical experiences with individual and/or collective control in co-located versus virtual dislocated environments poses new challenges as the types of devices, latency and the number of collaborators grow exponentially. While issues of scalability were discussed in our paper, techniques to address multiplayer interaction and identification of an individual’s musical agency in large-scale collaborative music making experiences have yet to be fully explored. Despite the advancement of new technologies, many questions still remain regarding the most satisfying pathways to virtuosity, expressivity, reproducibility and organization of musical output in a collaborative environment. Although a myriad of options exist for discrete versus continuous control to allow for interactive improvisation and musical transformation with a range of controller choices, the quality of collaborative engagement for amateurs and experts is still difficult to measure and evaluate. For novices, predictable control, intuitive mapping and connection between players are still paramount to the quality of the experience. For skilled players, higher levels of creativity, expressivity, and interdependence in a non-linear cohesive sonic environment are generally more important factors toward achieving musical satisfaction in a collaborative setting.

It is exciting to see that the range of collaborative experiences has dramatically increased since the Contexts of Collaborative Musical Experiences was written. Although the design principles we set forth were primarily developed and examined under the lens of collective engagement in a shared public space, we believe they are still relevant even as new technologies enable people to get together to enjoy music making in social networked contexts that were not viable at the time.

Expert Commentary: Social Engagement Before Bits and Bytes

Nick Bryan-Kinns

All too often as digital creatives we lose ourselves in the technological possibilities before us and forget the simple pleasure of engaging and being expressive with other people. To me, the key contribution of this paper is to turn this attitude on its head and to emphasise the value of the social experience of music making whether it is by novices or trained musicians. After all, music’s central role in society and social interaction predates not just computers, but also Western music conventions (Titon 1996).

It is striking that the attributes of collaborative music interfaces identified by Blaine and Fels are still relevant and applicable today. Indeed, this paper is often one of the first I recommend my students to read before they sit down to start their research projects. Conversely, the technology that we build our NIMEs with have changed radically since the paper was written. Instead of having to hand-code client-server systems to support collaborative music making, there are now easily accessible libraries for real time collaboration on-site, and across the web such as node.js. Similarly, instead of having to build bespoke microcomputer architectures and hardware to support tangible interaction with sound, there are now whole open-source platforms, such as Arduino, which can easily be used to create all sorts of wonderful interaction possibilities, let alone the interaction possibilities now offered by smartphones. The increasing accessibility and openness of hardware and software which can support collaborative music creation makes Blaine and Fels’ paper even more valuable today by providing a lens through which to view these technological advancements over time. For me, Blaine and Fels’ paper led me to think beyond the technology, and to explore what mutual engagement means between people when they creatively spark together (Bryan-Kinns and Hamilton 2009).

The work of Blaine and Fels sets out clear elements of the design of collaborative musical interfaces. What it does not do, though, is to provide mechanisms to evaluate designs in terms of these design elements. Developing reliable and easily deployable methods and tools to support evaluation of collaborative music interfaces is the next step to improving our social experiences with collective musical. Similarly, the two design elements of “Player interaction” and “Pathway to Expert Performance” are critical design elements for new systems, but are only briefly touched on in the paper. These two elements deserve significant research in their own right. For example, the sense of control and contribution to the collectively produced music (Player interaction) has emerged as a research topic in its own right, and likewise, providing appropriate scaffolding for expertise development remains a key question for NIMEs.