2001: Principles for Designing Computer Music Controllers

Abstract

This paper will present observations on the design, artistic, and human factors of creating digital music controllers. Specific projects will be presented, and a set of design principles will be supported from those examples.

Appendices

Author Commentary: NIME: 15 years Later. Wow!

Perry Cook

What a run it’s been. I could never have imagined that this paper might become one of my most cited and downloaded publications, but I guess it makes sense for a variety of reasons. Here I ponder some of those reasons, and give some thoughts on the time since.

First, it was the very first NIME, actually the ACM/CHI workshop, before the NIME organization and conference itself began running autonomous from ACM CHI. So the network effect of being one of the first papers helps a lot.

I won’t pretend that the information in this paper was particularly sage or epic by any means, but it did provide a good introductory reader and reference for a new curriculum area just getting started in music tech programs around the world. Long before NIME (5 or 10 years or more before) there were courses like Dan O’Sullivan’s Physical Computing course at ITP/NYU, and some other HCI Technology courses (like the multi-university San Jose State + CCRMA + Princeton + others course that Knapp and Duda (1995) got started with many of us back in 1996). But with the birth of NIME, courses like this could be re-focussed on music (the true goal for many of us all along), so having a NIME course made sense for many music technology and digital arts curricula.

This paper came before a lot of other cool things that happened in my academic and artistic life. Most of the experiences that brought about my 2001 list of NIME Principles came from working with NIME builders: at CCRMA, and with DanO, Bob Adams, Michael Brook, Geoff Smith, Dan Levitin, Bill Verplank, and others at Interval Research (Levitin et al. 2002), and Ben Knapp and others on our own HCI Technology courses (Knapp and Duda 1995). Princeton and my DSP + NIME curriculum brought me Dan Trueman and BoSSA, Ge Wang and ChucK, many (hemi)spherical speakers with Dan, Scott Smallwood, Curtis Bahn, Stephan Moore and others, the Princeton Laptop Orchestra (and many other LORks), Ajay Kapur and his Karmetik Robots and Machine Orchestra, Rebecca and Wekinator, many other great students, and lots of other NIME developments.

Finally, I updated the paper subsequently with new insights, revisions, and corrections based on changing technology or a different vantage point on my part.

My NIME 2007 Keynote at NYU, titled: “Principles for Controlling Computer Music Designers,” (Cook 2007) took a look at teaching a NIME curriculum, and early lessons from the Princeton Laptop Orchestra. So I added some new principles:

• 15. More can be better (but hard) (PLORk went from an initial 15 members, to 35!)

• 16. Music+Science is a great teaching/marketing tool (important in academic jobs)

• 17. The younger the student, the more fearless (they don’t know/care what’s hard)

My NIME 2009 paper, “Re-Designing Principles for Computer Music Controllers: a Case Study of SqueezeVox Maggie,” (Cook 2009) was about revisiting some older versions of my NIME instruments and my experiences with actually re-designing/re-building them. And of course, I changed one:

• 13. (b) Funny is often much better than serious (whimsical designs are not a crime)

and added even more new principles:

• 18. Redesign for backward compatibility (make sure you can play your old pieces)

• 19. Design (and pack) for post-9/11 travel (document, make sure things demo easily)

• 20. (a) Build a (new) copy, don’t trash the original (keep it around to compare), and (b) Build two or more if you can afford it. (one will invariably work better)

• 21. Wire and document for future surgeries (labeled 20 in paper, should be 21)

• 22. Build in diagnostic features and displays (labeled 21, should be 22)

• 23. Construct controller proxies (functional GUIs) (labeled 22, should be 23)

So one additional (personal) principle that I would add has to do with double-checking the numbering of lists in papers and sequels. Another principle might be that we never know where a piece of work, or publication, or small workshop as an adjunct to a larger conference, might lead. The ACM/CHI New Interfaces for Musical Expression workshop turned into something bigger than any of us might have dreamed.

I’m really happy to have been a part of the NIME history (so far). All indications are that we’ve still got a lot ahead of us.

Expert Commentary: Perry Cook’s Principles Still Going Strong

Marcelo M. Wanderley

The NIME 2001 workshop was a very interesting venue. A rather active musical controllers community already existed for several years in 2001mostly active at the International Computer Music Conference (ICMC), with key papers in the area already published before the NIME workshop (including many by Perry himself. See the electronic book “Trends in Gestural Control of Music” for a state of the art in 2000).² Even so, the NIME 2001 workshop was a unique opportunity to meet researchers directly working in this area, see demos of their work at the Experience Music Project in Seattle, and discuss possible ways to organize a research community around it. Several of the papers presented in the workshop became main references in our field, with several hundred citations each: Perry’s, Wessel’s and Wright’s (2001) and Orio et al.’s (2001), the two last ones with sequels published in the 2002 Computer Music Journal special edition (vol. 26, number 3) on NIME.

It was very interesting to re-read Perry Cook’s 2001 NIME workshop paper, as well as its NIME 2009 follow up paper where he reviews the original 2001 guidelines in light of newer developments. Perry’s 2001 paper belongs to a tradition of works from builders/performers (or performer/builders) who excel in both areas. Such works have the advantage of bringing to the community lots of personal, first hand performance experience informed by a strong technical/scientific component. In this sense, it is complementary to works by designers who mostly focus on technical aspects, who nevertheless collaborate with (multiple) artists when designing their devices. The advantage of the later group is that one can (hopefully!) more easily avoid one’s own aesthetic ideas about music and art and provide a larger, albeit forcefully less coherent, set of examples from which to derive guidelines. The study of both approaches, taking into consideration their own strengths and limitations, should lead to interesting (and informed!) designs.

It is amazing to see that in almost 15 years, many (most?) of its principles and guidelines still hold true. My favorites are (#4) “Some players have spare bandwidth, some do not” (true, how true!), (#2) “Smart instruments are often not smart,” (#1) “Programmability is a curse,” and (#6) “Instant music, subtlety later.” The last one ties well to Wessel and Wright’s principle: “Low entry fee, with no ceiling on virtuosity” (Wessel and Wright 2001).

Being part of the group who focuses mostly on technical issues (an example of a great paper about design is Bongers (2000)), I am less in agreement with principle #5 (“Make a piece, not an instrument or controller”), as I have seen many interesting instruments made without a musical piece in mind, but that subsequently were extensively used musically. The case in mind is Joe Malloch’s t-Stick (Malloch and Wanderley 2007), who in collaboration with composer/performer D. Andrew Stewart, developed a device that became an established instrument used in dozens of musical pieces (check Andrew’s impressive t-stick performance videos at vimeo.com).

I also take principles #10 and #11 (“New algorithms suggest new controllers” and “New controllers suggest new algorithms”) with a grain of salt, as I believe that they underplay the idea importance of mapping (between controller outputs and algorithm inputs). Complex mapping strategies can provide tools to use a variety of algorithms with the same controller (e.g. Hunt et al. (2002)), leading to interesting instruments. Furthermore, not all instrument designers are proficient in sound design and might just want to use algorithms they are familiar with, even when using different controllers (and conversely).

Finally, other principles seem to have become less useful with time, as for instance, battery life that has dramatically improved in the last 15 years (#8), making the use of cables is less necessary in instrument design (although still useful in many occasions!) (#9). In short, this is still a very useful, inspiring paper. Perry’s direct style is excellent and the review of his previous works until then is enlightening. Some of its principles will hold for a long time (forever?) and have become basic bricks to our community.