Keywords

1 Introduction

When we hear sound, it is all around us; the space this sound occupies is always present in one way or another, or is revealed to us in the listening situation. Space is revealed through the different attributes of the sound matter and is neither empty nor absolute. This chapter looks to map out some of the features and connections between sonic design and spatial features by drawing on Pierre Schaeffer’s theories surrounding the sound object and his system of classification and exploration called the typomorphology. This was described and laid out in his 1966 book on music theory Traité des objets musicaux: Essai interdisciplines (English translation published in 2017). This music theory, with its interdisciplinary approach, was not realised as a how-to guide for composers but instead was the culmination of a research project which sought to bring together issues in musicology, acoustics, philosophy, and psychology (Valiquet 2017).

Through his music theory, Schaeffer did not consider space as necessarily relevant in itself, but rather that time is where the object exists. This is despite earlier performance research that utilised spatial technologies like the potentimètre d’espace, a device used during the performance of Symphony pour un homme seul in 1950 to move sound between three loudspeakers (Holmes 2012). Schaeffer and colleagues did not have access to the technological tools we have today, and it is fruitless to speculate how the spatial parameter would be incorporated into his work. However, in the Outline of a concrete music theory, with Abraham Moles (Schaeffer 2012), the two authors defined “25 initial words for a vocabulary” (p. 191–194), where words 23–25 are defined as spatial music, static spatialisation, and cinematic spatialisation. Spatial music is any music “that is concerned with the localisation of sound objects in space when works are being projected to an audience;” static spatialisation is defined as static sources in space, locatable to a point; and cinematic spatialisation refers to “projection that makes sound objects move in space at the same time as they move through time” (Schaeffer 2012, p. 194). Building on this, Iannis Xenakis formulated concepts of stéréophonie statique and stéréophonie cinématique, referring to sounds that are distributed over loudspeakers as points or where the sound sources are mobile and moving, what Maria Anna Harley referred to as trajectories sonores (Harley 1998).

The French composer Edgard Varèse iconically referred to his practice of composition as organised sound (Varèse and Wen-Chung 1966), where all possible sounds could be of musical interest, expanding the possible scope of composition. This encompassed all ranges of acoustic phenomena as equal in “value” to the perceived limited scope available within the voice and acoustic instruments. Varèse used topological and spatial metaphors to describe his work of “shifting planes, colliding masses, projection, transmutation, repulsion, speeds, angles and zones” (Born 2013, p. 2). Not only has this discourse influenced the subsequent development of spatialisation approaches, but I will argue throughout this text that these metaphors have also paved the way for new approaches to understanding sound, to a practice of expanding sound to space. Varèse argued that with “the liberation of sound,” our conventional music notation systems would be inadequate to convey the new music, rather “the new notation will probably be seismographic” (Varèse and Wen-Chung 1966, p. 12).

Sonic design has been proposed as an “interface” for studying musical sound (Godøy 2010); however, from its basis in Schaeffer’s theories on the sound object, the contribution of sonic design extends beyond mere musical sound and into spatial features and significations. Schaeffer theorised the sound object as a basic unit of perception where it is “capable of making a rich set of perceptually salient sonic and multimodal features present in our minds” (Godøy 2018, p. 761). This is a proposal of the sound object as a multidimensional unit, meaning that it can contain multiple significations and features simultaneously and is an ontologically complex unit. How we perceive these different features depends on our intentional focus and what features we focus on when listening. This multidimensional unit can be described by its main features and broken down into sub-features and sub-sub-features where all the different feature dimensions have various values (Godøy 2021). When perceiving sound objects, we can access the various features of the sound through its mediation to us as a sign. When shifting our intentional focus to the features contained in the sound object, we cannot see the source, and this is no longer of any relevance, as we focus on what we hear and how we hear it. The potential references to an external sound event will still be evident, depending on our intentional focus. Likewise, “smoke is only a sign of fire to the extent that fire is not actually perceived along with the smoke” (Eco 1979, p. 17).

This chapter will contextualise these concerns and discuss how the core feature of sonic design and Schaeffer’s theories on the sound object can be extended to multidimensional entities of morphologies of space and as an informing element in spatial audio applications. Before discussing the typomorphological classification system, the central concept of anamorphosis, or warping, will be discussed in relation to the physical signal and the subjective perception of sound.

2 Linear and Non-linear Relationships

The relationships between the linear and non-linear are not always easy to define. Still, within the context of sound and audio programming, we can say that in a linear system, we can multiply a signal by a constant for amplification or attenuation of the signal; and in a non-linear system, we can multiply a signal by another signal, as in amplitude modulation (Smith 1999, p. 95). Linear relationships can be plotted in a straight line and divided into modular parts. Linear systems can be taken apart and put back together again, unchanged. Non-linear systems, however, “are not strictly proportional. One can think of them as having internal thresholds; when these thresholds are crossed, they switch into another mode of behavior” (Roads 1996, p. 887).

Non-linear relationships exist between the physical signal and the subjective perception of the same sound. These relationships are always present; we not only listen to “the sound itself” nor perceive it on its own; we also listen from a particular spatial position and perspective. Michel Chion emphasises that it is “not the psychology of the auditor that matters, it is the particular spot where the latter is positioned that does” (Chion 2016, p. 172). This points out that our experiences shift depending on our position, and we should be concerned with exploring the possible correlations between the physical signal and the subjective perception of the sound, specifically the relationship between the signal, the space, and the listener. This is especially pertinent given that the “sonic image emerges, therefore, as a concept that can integrate different listening approaches and provide an understanding of both the intrinsic and the extrinsic aspects of sonic experience” (Barreiro 2010, p. 36).

These piece-wise cumulative images (Godøy 2006), indicate that we piece together a sound and its behaviour in incremental steps. This becoming of sound perception can be referred to as what Norbert Schnell called an action–action relationship, meaning that any action is not isolated but is always part of an inter-action, the result of, belonging to, and becoming an interactive relationship (Schnell 2013). The relationships between the signal, the space and the listener exist between what Dick Raaijmakers described as “from the smallest sound to liquid form” (Raaijmakers 2000, p. 81) as well as the possible transmutations and morphogenesis of objects that can range from “dull matter, hard resonant matter, flowing liquid, bubbling liquid or steam clouds” (Roads 2015, p. 312).

Objects can have multiple significations, and as we shift our intentional focus to attend to different features of these multidimensional objects, we also shift our focus between the object as we hear it spatially and how it is situated in three dimensions:

The essential aim of spatialization, which is often confused with some strange myth of “spatial music”, is to improve the definition of objects through their distribution in space, since it so happens that the ear distinguishes two simultaneous sounds better if one comes from the right and the other from the left. We are not dealing here with a luxury added on to our hearing but something to facilitate it. Before even mentioning space and sound architecture, we should talk about the identification of objects and their coexistence. Where they are is of little consequence; it is what this enables that is important: an incomparably clearer, richer, more subtle perception of their contents. In the same way, binocular vision gives the third dimension and by putting things in perspective with each other allows us to judge their properties and relationships better. (Schaeffer 2017, p. 325)

Schaeffer referred to anamorphosis, or warping, as the possible non-linear relationship between the physical signal and the sound object, that could be characterised by irregularities that suggest a distortion of physical reality (Schaeffer 2017). This concerns the mapping of correlations between subjective images and the acoustic basis in sound. For example, temporal anamorphosis leads to “time warping” that describes how a “listener’s perception affords conclusions that do not concur with physical reality” (Landy 2007, p. 79).

Anamorphosis is a visual distortion that requires the viewer to be in a specific location to see the correct image(s); it is a technique to create pictures within pictures. One example is Hans Holbein’s painting The Ambassadors (1533), a much-cited double portrait of two unknown ambassadors with a still life. The painting features a smeared shape across the front of the painting. This shape reveals itself to be a human skull when viewed at a sharp angle from the right, an example of a memento mori.

Another example can be found in the work of Maurits Cornelis Escher, where the pictures within the pictures are accessible for the viewer from one position, depending on where we focus our gaze. His lithograph Waterfall, for example, depicts a waterfall and a waterwheel, where the water seemingly flows downhill after the waterfall, only to return to the waterfall, causing a feedback loop. This warping of an image indicates that it is the subjective perception, from a specific angle, that should be considered significant but not the only thing we should attend to.

The correlations between the physical signal and the heard sound are essential because what we think we hear is not always what we do, in fact, hear. The signal is a carrier of information, but it is not the information itself; it is a representation of information (Garnett 1991)—the physical experience of music is related to the physical vibration propagating through a medium before it reaches our ears. For example, in the fields of sonification and auditory display, the sonification process must be rooted in the data it presents, but what is perceived is still sound, from which we can extract information as we would with listening to any sound. The information contained in the sonification should be perceived by the listeners (Grond and Hermann 2014). The perceptual experience is the psychoacoustic feature attributed to how we make sense of what we hear, and the cognitive features surrounding the listening experience determine the structures we make of what we hear and what it means to us. This approach allows us to explore the deeper facets between physical signal propagation and our subjective perceptions.

The relationships between sounds and their perceptions imply the need for an empirical feature mapping between the percept and the signal (Godøy 2021); that is, our subjective perception of these sounds is considered the most important aspect (Godøy 2019). However, the acoustic features of sounds and their propagation through space are not irrelevant, so we should be concerned with studying the correlations between sound, space and listener. As part of his music theory, Schaeffer presented a framework for the classification and understanding of sound objects, the typomorphology, and this system lends itself to the study of both sound and space not merely for the evaluation of sound features.

3 A Brief Outline of the Typomorphology

In Schaeffer’s music theory, the typomorphology provides a framework for understanding transitions in sound perception, and likewise, it can provide a framework for understanding spatial transitions. The complexity of Schaeffer’s theories should not be underestimated, nor should the rigour in examining the sonic matter. Chion states that a summary of the types, classes, species, and genres of objects can be found in the Summary Diagram of the Theory of Musical Objects), which is a tool for investigation and not simply as a set of results. This, in turn, is further emphasised as: “the general procedure in this music theory is to move forward in a series of approximations rather than in a straight line” (Chion 2009, p. 100). Then, the general idea in this music theory is a series of approximations through a process of analysis and synthesis.

Analysis and synthesis refer to the systematic exploration of features. It is a method to understand the world by breaking it into smaller parts and looking at the possible interactions between the parts and their surroundings. This has been described by Jean-Claude Risset as analysis by synthesis (Risset and Wessel 1999). Analysis refers to decomposing something of varying degrees of complexity into smaller parts or elements. This also includes interactions and perspectives. Synthesis refers to the operations involved in putting these decomposed elements back together as themselves, as new configurations or through the combinations of interactions (Risset 1991; Wright et al. 2000). By drawing on the joint perspectives afforded by both anamorphosis and analysis/synthesis, we can investigate the typomorphology.

The typomorphology is a descriptive inventory that precedes musical activity; it is the initial “phase in the programme of musical research” (Chion 2009, p. 124). The typology is a “first sorting” according to the overall shape of the sound, and the morphology looks at the internal characteristics and features of the sound object. The tasks of the typomorphology are identification, classification and description, and it is divided into three parts (Chion 2009, p. 124):

  1. 1.

    Identification of sound objects (typology)

  2. 2.

    Classification by type (typology)

  3. 3.

    Description of characteristics (morphology)

Identification and classification of sound objects is a procedure which consists of isolating and cutting out sound from all possible contexts and then arranging the sound objects by type. This sonic examination is based on subjective judgement. It is done in terms of reduced listening and involves a temporary suspension of our knowledge about the world and the sounds we are listening to in order to access their features. The typology starts by identifying sounds into three different categories based on their dynamic envelope:

  1. 1.

    Impulsive

  2. 2.

    Iterative

  3. 3.

    Sustained

These three categories describe sounds by their dynamic envelope, where an impulsive sound is short and fast produced by striking an object. An iterative sound describes something with a rapid motion, which can be perceived as a stream of impulses. A sustained sound is something prolonged, with a reasonably steady envelope. This perception could be true when hearing an impulsive sound of, for example, a snare drum being hit or a glass breaking in a dampened room like a studio. However, a changing spatial context for different sounds would classify them into different categories. For example, this same impulsive sound played in a reverberant space would cause the sound to be iterative rather than impulsive. As Tor Halmrast demonstrated, the attack of a tone is lengthened due to reverberation, which masks its entrance into another, and, depending on the sound source, this creates an attack of the attack (Halmrast 2018).

Even here, at the very start of the classification of sounds, the spatial presence makes evident the sound’s relationship to something external to itself. For example, the measure \(RT_{60}\) (reverberation time) tells us how long it takes for the sound pressure level to drop by 60 dB (Howard and Angus 2009, p. 301). This is an easily understood parameter, but it says nothing about the materials of the room, the number of reflections, arrival times of these, or their strength. This causes rooms with the same \(RT_{60}\) to sound very different (Halmrast 2015). Figure 1 displays three impulses recorded in slightly different spaces. The first impulse was recorded in a heavily dampened space, where soft materials covered the walls, floor, and ceiling. The second impulse was recorded in the same room, with the door open, onto a concrete hallway. The third impulse was recorded in the concrete hallway outside the dampened room, which also has many intersecting corridors. The changes imposed on the sound by the surrounding space are clear. Indeed, when examining concert halls, David Griesinger found that the sonic background of a performance space can have unique timbral and spatial qualities and properties (Griesinger 1997), which introduces different timbral colourations to the sound. This would surely also be true for many other spaces.

Fig. 1.
figure 1

Three impulses recorded in slightly different spaces. a) An impulse recorded in a dampened space with a short reverberation time. b) An impulse recorded in the same space with the door open. c) An impulse recorded in a reflective hallway.

Full size image

After the initial identification of sounds, they can be classified into pairs of typological criteria, where they are used to give approximate distinctions between objects (Chion 2009, pp. 134–137). This includes its mass/facture, which describes how a sound occupies the spectrum and how its shape changes over time. Duration/variation describes how a sound is subjectively experienced and how it is experienced over time. Finally, balance/originality deals with the object’s internal structures.

The morphology, then, is divided into seven criteria of mass, dynamic, harmonic timbre, melodic profile, mass profile, grain, and allure (often referred to as gait/oscillation) (Chion 2009, pp. 158–187). The aim is not to identify abstract values such as pitch classes but to classify and understand sound in its possible diversities. This also extends into spatial features. Of these criteria, the most interesting in this regard are the two last, that of grain and allure.

  1. 1.

    Grain is a microstructure in the sound, which can be fine or coarse, and refers to the perceived surface of the object and its tactile texture. It can refer to a rapid gait or variation or an accelerating iteration. A rapid succession of impulses stops being perceived as impulses but becomes a continuous sound with a characteristic grain.

  2. 2.

    Gait (a suitable translation of the French word allure, which means to walk, or a way to walk) refers to an undulating movement or fluctuation of sound objects, which can also be described as an oscillation. The oscillation of gait can both be in terms of duration and motion. The gait of a sound can be seen as a “signature” of its source.

These criteria are used as descriptions of sound. Why should we use such a system to describe space when we already have the tools defined by acoustics at our disposal? Indeed, the typomorphology does not afford us a description of space but gives us the means to describe behaviour in space. When we hear a sound emitted in an enclosed room, we hear a single fused sound that consists of the direct sound emitted by the source, along with a series of reflections from different surfaces in the room. The ratio of direct-to-reverberant sound is vital in distance perception of a sound source, as well as early reflections, high-frequency attenuation, and air absorption (Moore 2003).

Perception of sound quality, sound colour, or timbre is not solely dependent on the sound “itself,” the spaces also contribute. Different rooms sound differently, and “the background can have its own spatial and timbral properties” (Griesinger 1997, p. 725). In a concert hall, the first reflections, primarily through interference by comb filtering, lead to a change in tone colouration and “image shift” (Barron 1971). Likewise, our spatial perception can be influenced by echo disturbances, shifts in the image of the apparent source, shifts in spatial impression, and different modifications of timbre (tone colouration) as functions of differences in intensity and arrival time (Kendall 1995).

The influences of, and changes to, sound given the environment it propagates in shows that the spatial situation is an essential factor in understanding the heard sound, the sound we perceive in the spatial listening situation. This is important for considering spatialisation approaches, which are not merely concerned with panning a source around a set of loudspeakers but should consider the entirety of the listening situation. When examining space from the position of a music theory concerned with manipulating sound materials, it poses a series of problems which are not obvious to resolve. The following section will discuss different approaches to experiencing space from the basis of psychoacoustic perception before examining possible approaches to working with space from a perspective of spatialisation.

4 Perspectives on Space

The classification system represented by the typomorphology provides the listener with a framework for exploring the many features of sound objects. It does not consider space as some abstract entity but analyses sounds for their features, shapes, and motions. As we saw earlier, Schaeffer did not consider space relevant in and of itself, but extending the typomorphology in this way will provide us with a richer set of tools for evaluation and classification. Indeed, the perceptions of spatial environments depend on the listener’s accumulated knowledge of the physical and external world:

When sensing a spatial environment, an individual builds a cognitive map of space using a combination of sensory information and experiences accumulated over a lifetime. The cognitive map of space in our consciousness is subjective, distorted and personalized - an active and synthetic creation - rather than a passive reaction to stimuli. (Blesser and Salter 2009, p. 46)

This construction of the cognitive maps we use to sense spatial environments reflects the concerns in emphasising anamorphosis to describe the relationships between sound and signal. It aligns with the message from Schaeffer’s musical research that it is through our subjective and attentive perception of the world and the sounds contained within it that we make sense of what we are experiencing. For identifying a sound object, the “identification is done by reference to a higher level of context which includes the identified object, as an object in a structure” (Chion 2009, p. 61). It becomes clearer when examining the different criteria from the typomorphology, that sounds have a relationship to the external world, and it is the sound’s morphological criteria that provide us with clues about how it existed spatially and how we can make it exist spatially.

At the opening of this chapter, Varèse was cited by using a series of metaphors to describe the behaviours of sounds in space. These feature categories lend themselves well to the transformation of sound materials but also as space descriptors. These same concerns have been formulated by Jean Petitot, in relation to morphodynamic models and how they unfold as bifurcating, non-linear dynamic systems:

The phenomenological description of sound images, sound structures and sound organizations is very diverse; it includes forms, figurative salience, clear and fuzzy contours, attacks and fronts, not to mention deformation, stretching, mixing, stability and instability, rupture, discontinuity, harmonic clouds, crumbling and deviation of figures and so on. (Petitot 1999)

The bifurcations Petitot describes are related to both Varèse’s topological and spatial metaphors of colliding masses, shifting planes, projection, and transmutation, and to Roads’ dull matter, hard resonant matter, flowing liquid, bubbling liquid or steam clouds referred to earlier. A bifurcation is a point where something divides into two parts (or branches) and is used as a model of transition of features (Strogatz 2015). Metaphors are used to describe, experience, and understand something in terms of something else (Lakoff and Johnson 2008).

When we encounter sounds and sound experiences, we use metaphors to describe their features; for example, a sound can be described as “smooth,” “shrill,” “rough,” “boxy” and the like. Metaphors can help composers and listeners to describe something vague as more tangible. The use of metaphors as a language to describe perceptions of sound can be a means of explaining the mental image of a sound (Porcello 2004) and even by adopting metaphors from other fields as a means of sensory evaluation, such as using terminology from the wine industry to describe features of concert hall acoustics (Lokki 2014). In the wine industry, the aroma wheel is a systematic way to discover the various flavours and fragrances found in wine and, looking beyond personal taste, the wine industry has established an overall characteristic of wine. However, with the available terminology and attributes to describe spatial features, this is not found in the same way with sound perception.

We can often refer to objective parameters, as defined in ISO3381-1:2009 as a guideline and standard for room acoustic measurements. However, this guideline does not discuss the subjective perception nor preference of listeners (Lokki 2013). The subjective perceptions of concert halls are difficult to measure, and this highlights the need to go beyond the impulse response measurement and standard criteria (Halmrast 2015) to focus on the perceptual consequences of frequency-dependent phenomena in musical instruments and human spatial hearing (Lokki 2016).

Yet, adopting metaphors for describing sounds and space can lead to inaccurate and conflicting descriptions among listeners. The different spatial attributes of sounds are, as we saw earlier, grounded in real-world experiences, sound perception, and localisation, abstraction of objects, relationships between objects, and the perception of space through the mass and size of objects. This can be described by the following four points:

  1. 1.

    Perception of sound as a whole, through object cognition and smearing in time and space

  2. 2.

    Immersion in the sound, the perception of not only the listening space but also the inherent spatiality of the sounds and their external references

  3. 3.

    The perception of multiple locations and distances and the proximities between sounds, is essential for the understanding of the relationships between sounds in a space

  4. 4.

    The perception of space through the size and mass of objects

“Real-world” indicates that which can be sensed from our surrounding world, either directly through our biological sensory apparatus or through microphones, sensors or other data collection methods. Through “sounds as a whole,” we gather some form of impression of the supposed origin of the sound and its spatial context.

To describe the perceptual cues and the mechanisms of human sound localisation, we can use criteria defined through psychoacoustics to aid in the description and classification of sounds. The dimensional features in spatial sound are impressions in terms of spatial extent (width, depth and height), distance and direction, and immersive features such as presence, envelopment, and engulfment. In their normal usage, these attributes describe spatial and musical percepts and how the human mind makes sense of these experiences. These attributes can also provide us with insights into how the identification, classification, and description of sounds can be made through the typormophological framework.

Akin to how a sound object was described at the opening of this chapter, as a multidimensional unit containing multiple significations and features at the same time, experiencing sound and its acoustic correlate is also characterised by an array of multidimensional features, as exemplified above in the wine and food industry. Within acoustics, there is a wealth of terminology for describing space and spatial experiences but no agreement on many features. As an example, spatial impression is used to describe whether a space is perceived to be large or small, and spaciousness describes whether we are in a large and enveloping space (Griesinger 1999). The terms spaciousness, spatial impression, and envelopment are interpreted variably in the literature, and spatial impression has often been used as a “cover all” term (Rumsey 2002). Several researchers equate spaciousness with apparent source width (Griesinger 1997), but spaciousness has no bearing on the perceived size of the source, “a concert hall can be spacious, the reverberation of an oboe can be spacious, but the sonic image of an oboe cannot be spacious” (Griesinger 1997, p. 721). The perceived spatial impression is dependent on lateral reflections between 125 Hz and 1000 Hz. It is a function of the performing level and will be higher with larger ensembles (Barron and Marshall 1981). The combination of early and late arriving energy determines the magnitudes of spatial impression, apparent source width, and listener envelopment (Bradley et al. 2000). If reflected energy arrives within 50 ms of the end of the sound event, it is perceived as a small room (Griesinger 1996). However, to explain spatial impression, both the frequency and level-dependent aspects of the music that arrives at the listener’s ears have to be linked.

These differences in terminology and lack of agreement on perceptual attributes can be a source of inaccurate and conflicting descriptions. Still, they are salient features that can be used to further develop the typomorphological framework and its potential spatial features. They are also informing elements in spatial audio applications. The next section discusses two approaches to working with space from a practical perspective and offers methods for thinking about both sonic and spatial design.

5 Approaches to Space

Marije Baalman differentiates between techniques and technologies (Baalman 2010). Techniques are descriptive of a compositional process, while technologies are descriptive of panning, speaker arrays, encoding/decoding functions, and so forth. When thinking about spatial audio or spatialisation approaches, it is usually limited to panning sounds between multiple loudspeakers. Rather than maintaining this relationship of formal properties to describe sound in space, we can draw on the typomorphology to explore further how sonic design can be used for spatial features.

Drawing on the methods discussed so far, two different but related approaches to designing space will be presented. Sound design involves the construction of sound worlds that exist in complement to a visual component, in this instance designing space will refer to the construction of holistic spatial scenes. Recordings made in a forest can evoke a sense of place and particularly a sense of depth (Westerkamp 2002). To gain acoustic knowledge of this space we can record impulse responses to replicate the acoustic presence of a forest and present it in a concert hall. It is common to be mindful of the foreground and background as important components in creating depth in a spatial scene (Lennox et al. 2001). Usually, also time-based processing effects like reverb are used to create larger spaces for the sounds to exist in.

We will yet again return to Varèse’s topological and spatial metaphors, which he described as “shifting planes, colliding masses, projection, transmutation, repulsion, speeds, angles and zones” (Varèse and Wen-Chung 1966). In what he termed “zones of intensity,” Varèse essentially described spatial experiences and organisation of sound materials, “these zones will be differentiated by various timbres or colors and different loudnesses” and “these zones would appear of different colors and of different magnitude in different perspectives of our perception” (Varèse and Wen-Chung 1966, pp. 11–12). Not only does this description put into perspective the concerns surrounding the realisation of Pòeme électronique at the Philips Pavilion in 1958, where the piece was spatialised over 400 loudspeakers, but also prefigures much of the technological advances made in spatialisation technologies for music, sound art, and film.

Fig. 2.
figure 2

A sparse, point-cloud reconstruction of a forest clearing, with trees seen as extending vertically around a central point. The example is made using photogrammetry, a technique where patterns are recorded and interpreted through photographic imagery.

Full size image

As with the metaphoric descriptions discussed so far, these features open a multitude of opportunities for how we can explore and analyse space through spatial audio applications. Many modern technologies for spatial audio emphasise a point-source approach, where an individual sound is placed in space as a single point. In the real world, sound does not exist as a point. The sound produced by a source will propagate outward in the surrounding space, and we will experience different frequency reflections and time decays from a series of surfaces in the space surrounding us. All sound sources have complex radiation and directivity patterns, and these complex patterns, combined with a potentially complex set of reflections from the surroundings, illustrate that a single point in space will not suffice as an element of spatial, sonic design.

To overcome this problem, one can design space based on photographs or personal experiences. However, by drawing directly on the physical makeup of the space to be constructed, various 3D scanning techniques can be used. The resulting point-cloud reconstructions can be used to build 3D maps of the space as a basis for developing sonic and spatial design, and through this construction, determine how sound behaves. Point-cloud reconstructions can be made through structure-from-motion, also called photogrammetry or LIDAR scanning. The resulting data collection can be used to build a 3D model which represents the physical constraints of the space. In the instance illustrated in Fig. 2, a point-cloud photogrammetry reconstruction is made of an area in a forest, showing trees extending as vertical lines around a central clearing. This illustrates how point clouds can be used dynamically to navigate a space and to create density maps of sound motions. This approach to spatial audio draws on principles from sonification and its uses of making data available through sound (Hermann et al. 2011). In sonification, the data defines and drives how sound behaves spatially and is concerned with the quality of the sounds and of the contexts to which they belong.

By constructing a 3D model of a space, sound behaviour, reflection, and diffraction can be modelled, giving the sonic designer more direct control over constructing a perceptual significance, that is “to describe the ‘behaviour’ of sounding objects in and through their local environment—this is not just the case of Doppler effect, as it includes the timbral changes due to comb-filter effects as the early reflection patterns change with movement” (Lennox et al. 2001, p. 8). The possible applications to games, virtual reality, sound-field reconstruction, and composition should be clear.

This can also be explored from a “non-real” perspective, where a point cloud can be randomly generated and used to determine densities and motions of sounds in space. In the instance displayed in Fig. 3, the number of points is randomly generated and the distances between them are determined by a rule-based system. This approach is similar to spatial swarm granulation (Wilson 2008), yet the points in this instance are not necessarily bound to granular synthesis processing techniques. For example, each point in the cloud can represent a filter, an impulse response, a sound or simple delay points where a sound is stretched as it moves past. The behaviour of such a point cloud can be determined using the popular Boids algorithm (Reynolds 1987) or as an approach to timbre spatialization (Normandeau 2009) or spectral splitting (Wilson and Harrison 2010), where each point represents a spectral bin (Kim-Boyle 2008; Torchia and Lippe 2004).

Fig. 3.
figure 3

A randomly generated point cloud which can be used to create density maps and motions with no basis in the real world.

Full size image

Rather than relying on the panning of individual sources bound to different speakers, these approaches mirror some of the theoretical concerns discussed so far in this chapter. Consider gait (allure), the morphological criterion which denotes the fluctuation or undulation of a sound. In Schaeffer’s system, this referred to characteristics of the sound itself, but gait can also be used to express how a sound moves through space. In sonic design, the feature space is not fixed, rather it is a field of possibilities (Godøy 1997). The grain quality of sound, refers to the perceived surface and tactile perception of a sound and could be extended into describing the perceived surfaces and reflective qualities of a space, as exemplified in the forest point cloud. Indeed, also the typological criteria of mass/facture, which relates how a sound occupies the spectrum and how its shape changes over time, can here be given spatial relevance. The density and distances between the points in the second example can dynamically be changed in response to how a sound changes over time.

Viewing the approaches to space sketched out here in relation to the psychoacoustic attributes discussed in the previous section widens the potential feature space as described through sonic design. Where gait, grain, impulsive, iterative, sustained, facture, duration, and variation describes the inner features of a sound, spatial impression, spatial width, apparent source width, and many other psychoacoustic attributes provide us with a salient framework for extending how we classify sounds and their spatial features.

The morphological visions of both Varèse and Petitot can in this way be given a renewed relevance and context in terms of spatial understandings of sound. Using methods of data extraction and reconstruction can create flexible models of motions and densities of how sounds move and behave. These approaches join Baalman’s differentiation between techniques and technologies with that of sonic design, where we can work directly with the different values of the features, sub-features, and sub-sub-features within the multidimensional framework. Through piece-wise cumulative images, we piece together a sound and its behaviour in incremental steps, including its spatial qualities.

6 Conclusion

Sonic design and the system for classifying sounds, the typomorphology, extend our understanding of the interplay between sound and space. The morphological descriptions described by Schaeffer in the Treatise on Musical Objects provide a rich tool-set to pursue and understand these perspectives from artistic and scientific perspectives.

Spatialisation designs are often made on purely technical grounds where individual sounds are panned from speaker to speaker, layered, and moved in and out of specific densities. However, these approaches are often considered “after the fact,” when the sounds are made; what is left is purely a presentation format. Many current technologies for sound spatialisation emphasise a point source approach, where individual sounds are panned as points with no consideration for the remaining context. The approaches outlined above, with roots in a morphological description of sound motion, density, and presence within a spatial context, sketch out an open-ended approach to building spatial scenes. This approach draws on what was earlier referred to as an action–action relationship, where actions are always part of an inter-action. This approach reflects the methods of sonic design, where the criteria are subdivided and sub-subdivided in a top–down, subjective exploration of feature categories.

The sound object has been seen by many as a significant part of musical experience, both as a tool for understanding and creating music. In this chapter, by drawing on the methods of analysing and categorising features of sound objects, this has been extended into spatial dimensions and uses. Through this expanded focus, and by drawing on the rich and varied attributes of acoustics and psychoacoustics, there will most likely be many more salient features to identify as we are now not solely considering musical sound but spatial presence as well.