Keywords

1 Introduction

Working with embodied sonic design, I consider how the method defines the product. My work aims to create a novel sensory experience where sound, image, and movement converge. I create immersive sound installations using multi-sensory stimulation and by generating an awareness of the body in space through proprioception. My sound works are created using body movement in unconventional ways. This chapter is dedicated to describing the way I work with sound using an embodied process—outside of instrumentation, vocalization, or electronic sound practices—and prioritizing whole-body integration. I find sound to be an effective artistic medium to profoundly engage the body. In addition, I integrate conventions of film and dance. My artistic research, along with an investigation of embodiment and perception, have paved the way for the experience of my work to be that of listening to see, seeing to feel, and feeling to hear. This chapter is a record of how I work to depict physical presence through sound.

Rolf Inge Godøy’s work has encouraged my investigation of the relationship between sound and motion as an area of serious study. Examining sound–motion similarity in musical experience, defining sound tracing, conceptualizing sound–motion objects (Godøy et al. 2016), and developing the Sonomotiongram technique (Jensenius and Godøy 2013), have inspired foundational concepts essential to my practice-based research. What follows is an artistic reflection on sound, movement, and embodied sonic design as an exploratory practice for generating and sensing the details of sound more deeply. Through embodied sonic design, I consider the effect movement has on the recording and playback of sound as well as the experience of the listening body and how we understand differently the implications and the circumstances of sound as presented.

This chapter will detail the trajectory of my artistic research, which incorporates movement improvisation, theories of perception and embodiment, as well as research on the relationship between sound and movement. Through my artistic research, two embodied sonic design techniques have emerged: sound shadows, a source-blocking technique that features the presence of a moving body through the absence of sound, and embodied binaural spatialization, a recording method that captures the perspective of a moving body in a choreographed sonic experience for the listening body. Kinesthetic empathy clarifies the potential for embodiment and makes way for a different sensory process, which I explain through the sound installation the Presence in my Absence, a felt experience in three-dimensional space. I explain how I work with sound perceived as a moving physical form and reflect on the creation of the third movement, a work that explores the kinesphere of individual listening bodies. To close, I position my continued development of these techniques for embodied sonic design as a hopeful practice, inviting different perspectives and lived experiences to become the subject of my work.

2 From Theory to Practice-Based Research

My intention is to engage the audience as a whole, that is, to integrate sensations from the body. I use movement to produce a sonic experience for the listening body. Along with analogue-adjacent processes, I use the movement of my body to physically manipulate and record sound. In sensing movement, I am inviting the audience to situate their individual bodily perspective within the work, allowing the lived experience to temper the work in the absence of visual representation. In order to approach the medium of sound differently, I employ a filmic perspective, and I find it helpful to visualize my sound experiments through descriptions using the conventions of filmmaking. Furthermore, pitch, dynamic, attack, and timbre work to determine movement in a way akin to highlighting, shading, and other visual effects used in animation (Clarke 2005, p. 73).

My first experiment compelled me to create the experience of movement by the body, self-motion. I considered what kind of sound–movement would most clearly illustrate self-motion, as opposed to movement around the listening body, and endeavored to create the sensation of spinning. I placed a recording device, standing in for the listening body, in the seat of a spinning office chair with a set of stereo speakers playing an ethereal-sounding piece of music nearby. The speed the chair was spinning, and the rate at which the sound panned from the left to right channels depicted a continuous motion. The same decay rate I would have sensed through inertia also helped to establish the effect.

Besides speed and realistic motion, I considered other factors influencing the spinning effect. First, the Doppler effect alters a sound’s perceived frequencies based on its motion relative to the listener. I understand this as the “bending” of sound waves as they move through space, resulting in a change in pitch. The Doppler effect is responsible for the perception of speed and the location of sound. I charted these variances by moving a small speaker towards and away from my ear, then from a recording device, and found the Doppler effect illustrated the quality of my movement. Listening live through headphones connected to the recording device, I recognized that I could vary the pitch with a subtle increase or decrease in speed and at certain points in space. Refining the desired pitch modulation was an act of precision that I reproduced through movement and muscle memory rather than auditory feedback. Using different sounds and musical samples throughout these trials also demonstrated that certain frequencies compounded the effect.

Convolution resonance, the unique resonance of sound in a specific environment, works with resonating sound waves to create realistic sound environments based on features of that space. I was struck by how the architecture I moved through while recording could be sensed while listening back. I could describe the feeling of the space even if I could not completely determine it. The environments I moved my body in were further situating the listening body. The sound of the music playing in the room aided the spinning effect, situating the sound source as stationary in a specific environment rather than spinning around the head of the listening body.

I have found that sound panning left and right alone is enough to engage my body. This technique is used in Autonomous Sensory Meridian Response (ASMR), a sensory practice where audio recordings, sometimes paired with video, can create the sensation of live action. Some listeners experience physical sensations or “chills” when listening to ASMR (Lochte et al. 2018). In music studies, this effect is referred to as “musical frisson” and can affect skin conductance and the release of dopamine. ASMR practices led me to use in-ear binaural microphones to create a spatial relationship between sound and the listening body and to think through a three-dimensional sound experience.

Our systems of perception and the sites they occupy in the body are important points of inquiry in the study of sound and embodiment (Clarke 2005, p. 75). The presence of both the vestibular organs, responsible for balance and orienting the body in space, and sound reception in the ears is of great interest to my work, although I do not investigate the biological relationship within this research. I treat the listening body as a whole unit.

Two techniques for embodied sonic design emerged through my practice-based research: sound shadows and embodied binaural spatialization. Sound is an indicator of movement and can depict the conditions of its creation (Clarke 2005). The effect of music can be drawn from its performance: the force, speed, and intention of the musician, among many other factors (p.75). I use these sonic design techniques to create sound installations that encourage motion perception in the listening body.

3 The Role of Movement and Improvisation

I approach sound as it relates to the body: as a vibrational experience and as spatial composition. My work is musical but should not be considered as music in a traditional sense. Further, I use the term movement rather than dance to highlight its transference onto different media. With a background in contemporary dance, I have trained in movement improvisation practices for years. My movement practice now informs how I approach the listening subject through embodied sonic design. Dance is a discipline that explores codified movement, interpretation, and improvisation as expression and performative communication. I use movement to generate ideas, facilitate sensation, and bring attention to the present moment. In my artistic practice, I create movement with sound and light and incite a sensation of dancing in the listening body; the work represents the effects of dance but not the act of dancing itself.

To improvise is to act without preparation, to proceed intuitively. I engage in movement improvisation in conjunction with the generation and manipulation of sound. It is integral to my research and is both a practice and a process. Susan Kozel, a contemporary phenomenologist, describes Maurice Merleau-Ponty’s pre-reflection and hyper-reflection as bodily experiences (Susan Kozel 2008, p. 23, as cited in Merleau-Ponty 1989, Part Two, Chapter 3). The pre-reflective state is an individual’s response that precedes judgement, like a reflex. Kozel deems pre-reflection to be a “challenge to language and logic” within the “domain of corporeality” (Kozel 2008, p. 21) because of how we experience it, viscerally. She uses the term hyper-reflection to describe a looping process where the observer considers their own role while in-action (Kozel 2008, p. 22).

Improvised movement and sound emerge non-virtuosic in a pre-reflective state. I suspend judgement by thinking through my body, becoming aware of my actions after they occur, and continuously acting before I have a chance to reflect. It is important to me that the pre-reflective experience is embodied by the listening body. Movement improvisation brings me to a state of action where I think only through my body. By recording improvised movement—as audio or video—I am layering it onto the present moment, making the process for both of my embodied sonic design techniques hyper-reflective.

David Borgo describes music improvisation as an engagement in complexity; ideas are disordered and conflicting but are of little consequence to the musician’s life beyond playing (2006, p. 22). Borgo suggests that engaging in this kind of play can increase a person’s capacity for discordance and, perhaps, discomfort. If sensing my movement improvisation transmits a pre-reflective state, perhaps it can empower the listener with the ability to move through uncertainty.

4 Perception is Art

Barbara Tversky writes about how action and perception are intrinsically linked, crediting much of our understanding of the world to how we move (2019). The reason I aim to engage the senses through movement is because it makes us think differently, offering the listening body a new state of mind. Through embodied sonic design, my sound recordings decontextualize, abstract, and reframe the auditory experience. This is how I break down the content as sensory information, removing it from its circumstance so that it may be taken up differently through the senses.

There is potential for greater sensitivity, and therefore understanding, in how we absorb information from the world around us. I examine affect by initiating effects on the body through listening to recorded sound (as opposed to the performance of live sound) to sustain the act of perceiving and to repeat the experience of listening. The sound works I create are ambiguous, recognizable by the senses, but resisting definition. In abstraction, we create our own meaning—much like we note pathways and patterns in space by simply tracking movement. The movement of sound delineates spatial information, which we consider in relation to ourselves.

Audio recordings act as context in my work, framing the experience. Once established, these recordings are an experience to respond to rather than a product of the moment. The concept of “putting thought into the world,” which Barbara Tversky (2019) suggests we do through language, gesture, and graphics, allows us to “transcend the here and now” (p. 99). An audio recording is a permanent and repeatable experience that we can build on. This is also the path of knowledge production.

Recordings offer a state of removal from an immediate event, which welcomes interpretation, reflection, and a moment to shift from an external to a more internal sense of awareness. The difference in how I work with sound is that the listening body must track perceived action: listening for the changing location of an invisible moving sound source requires a heightened sensitivity. Making sense of something through a collaboration of the senses focuses attention on our relationship to sound. This type of gestalt perception is something different, something unique to the experience. I call this kinesthetic projection.

Recognizing the process of perception is important here. I have likened my work to optical illusions, where—by design—something is perceived differently than it appears in reality. In realizing the discrepancy, we learn something new about how our eyes work.

5 Embodiment

Embodiment theories deem our corporeal experiences as rich sites of information to aid in the understanding of ourselves and others. The term embodied cognition acknowledges the sensory, bodily information of the cognitive process. Treating something outside of ourselves as if it were our own motivates my work. For me, feeling is essential. I am preoccupied with embodiment being the experience of sonic design, and I employ a phenomenological perspective when it comes to evaluating the perception of sound installations. Julie Herndon (2022) writes of embodied composition as a musical practice in which the creation of sound relates deeply to the body (p. 1). Herndon notes the relationship between movement and sound to increase a sense of connectedness through “internal and external awareness,” derived from this practice (p. 6).

Hearing accommodates simultaneous activity. We can consume sound with different levels of engagement; our bodies can be in any orientation to the source, and we can be in almost any physical or mental state and continue to listen. This allows us to connect our immediate personal experience with what we are hearing. The nature of recorded sound encourages an imaginative engagement, and my embodied sonic design techniques invite the listening body’s unique perspective. Democratizing experience in this way leads to a more sensitive account, furthermore, embodiment is part of the meaning-making process. With my work, I encourage meaning-making on an individual level by engaging the body and its multiple senses.

Embodiment through sound is not a novel concept. This is a large part of what makes us move to music, for example. Low-frequency bass sounds help to exemplify embodied listening here; they are usually felt more in the body than heard through the ears. Beyond music and entertainment, religious traditions and healing rituals that participate in chanting, singing, music, or sound use the affective qualities of vibration as a means of transcendence and worship. People also use sound to change their state of mind outside of faith practices. Listening to music or recordings of rainfall or birds singing can change our mood or promote relaxation. Binaural music is an emerging genre that uses separate audio tracks in each ear to achieve entrainment, synchronizing brainwave frequencies to achieve different mental states (Lochte et al. 2018). These diverse sound applications demonstrate different embodied relationships to sound.

6 Audible Kinesthetic Empathy

Watching movement incites a visceral, kinesthetic response (Wood 2015). When we watch others move, our bodies respond empathetically. We recognize movement by feeling as if we are executing the same movement. Barbara Tversky’s (2019) investigation of mirror neurons, and entrainment, explains simply that “body-to-body communication is more direct than word-to-word” (p. 62). Here, direct refers to explicit and instantaneous by way of a different mode of cognition. Tversky builds on the theory of motor resonance, which is how movement is recognized through our own movement patterns. As motor resonance suggests, the body recognizes sound by the action that produced it (Godøy et al. 2016, p. 4). Godøy’s research on sound–motion similarity in musical experience measured how different people moved to music and found culturally entrained responses. This inspired the sound-tracing paradigm, studying how the dancing body moves in unison with musical attributes.

Kinesthetic empathy is key to the embodied perception of movement. Tversky asserts that abstract thought—essential to activities such as problem-solving, creation, knowledge production, and the conceiving of possibilities—depends on an ability to think in spatial terms (Tversky 2019, p. 41). As Barbara Tversky uses the term, spatial thinking indicates mental reasoning through movement visualization. The example she uses is of mental rotation, where determining whether a distorted letter F, placed next to a correctly oriented one, is a rotation or mirror image of the correct position. Mental rotation theorizes that mental visualization moves the figure to reorient it properly and then recognizes the discrepancy (pp. 48–52).

I propose that tracking the movement of sound through space might also create a kinesthetic reaction. This is similar to how watching movement creates reciprocal activity in mirror neurons. As an example, a binaural audio track that mimics a swaying pattern from left to right might affect the listening body as if something is actually swaying in front of them. At this point, it might make sense to deem my practice as motion sonification. However, it is not the movement that is being translated into sound, movement is instead made apparent through its effect on sound. I use kinesthetic empathy here to create more dimensions in listening. The audibility of kinesthetic empathy enables other interpretations of sound, as different semiotic representations are embedded in performance.

The way certain music makes us want to move our bodies and the various translations of affective information produced through kinesthetic empathy, ASMR, and musical frisson demonstrate an ability to embody what we hear. The sensation of physical engagement from the perception of movement is what I aim to achieve through embodied sonic design.

7 Sound Shadows

I developed sound shadows as a technique for recording and presenting movement using sound, illustrating the effect that movement has on the environment. The sound shadows are created with a source-blocking technique using the audible interference produced by my moving body, partially obstructing the reception of sound by a recording device. Figure 1 represents the sound shadow recording configuration with a sound source positioned opposite a recording device with enough space between the speaker and microphone to capture the full moving body. The moving body between the sound source and the recording device blocks some of the sound, indicated by the arrows moving to the right of the image. This is akin to the way a body produces a shadow as it blocks light from a surface. The sound shadows are distinguished as the sound waves are eclipsed by the body. The conjecture is that during playback, the sound shadows could perceptually rematerialize the moving physical form blocking the sound during recording.

Fig. 1.
figure 1

Diagram representing the recording configuration used in the sound shadow embodied sonic design technique

Full size image

My sound shadow technique is similar to the sound installation remnant by Miller Puckette and Hagan (2019). Puckette and Hagan experimented with the effect physical bodies had on soundwaves and aimed to represent the absent bodies by recreating the effect their presence had on the sound and the environment. The shadow produced by bodily interference was measured, as well as the reflection of sound coming off the body, which was described as “scattering.” The two describe the process as “making the absent bodies themselves audible as acoustic reflections and shadows” (p. 1).

remnant used four musicians who played alto flute, trombone, piano, and percussion. My sound shadow technique uses white noise, sound with an equal amplitude distribution in all frequency bands. White noise fills the environment from veritable sonic events and punctuating sounds, which makes it effective as “noise cancellation.” I use white noise primarily as the wall upon which my sound shadows appear. This is a development of both Doppler effect explorations (altering sound by shifting its relative position) and convolution resonance (understanding conditions of a space based on the containment of the resonance).

Creating sound shadows is as much about finding the perfect backdrop to feature the movement occurring in the foreground as it is about moving in a way the body can sense. Puckette and Hagan recommend the use of a high-frequency sound, which makes distinct the difference between interference and clear transmission. I have found that blue noise—a type of noise with more high-frequency content than white noise—enables the differentiation between motion and stasis, and it tends to focus attention on the movement once the white noise is established as background noise rather than the main feature of listening.

Through exploration, I have found that sound shadows work best with short, consistent actions that can be transposed onto different body parts. Jumping, swaying, swinging, shaking, and approaching and departing—either from the sound source or the recording device, at varied speeds—were movements that translated well through sound. Here I should clarify that these movements should not be seen as gestures. The concept of the musical gesture brings in theories on body movements related to the shaping of sound with a communicative intention. While using the term gesture to describe musical effects “surpasses the Cartesian divide between physics and the mind” and encourages an embodied understanding, the movement I use is not coded in this way (Jensenius and Wanderley 2009, p. 19). My work attempts to engage the body without triggering an analytic determination of what is transpiring through language or gesture.

The best scene for designing sound shadows is one where the person can move freely within the audible “frame”—the area of clearest reception between the sound source and recording device. This leaves enough space for the background noise to come through. Ideally, the movement crosses at least half of the audible frame, so if the movements are small, such as intricate movements of the fingers, they should be performed close to the recording device. The scale of the effect depends on the distance between the moving body and the recording device rather than between the moving body and the sound source because, again, the recording device represents the perspective of the listening body. Small details should be close so they are in clearer “focus” the way more detail appears in the foreground of an image.

The process I use to create sound shadows recognizes the body as a tool for working with sound. The speed and intricacy that the moving body can express enables non-musicians, like myself, to produce soundscapes that would otherwise require musical training and technical knowledge. My embodied approach works through more intuitive expression and can be thought of as a new mode of sonic design.

8 360-Degree Sound Shadows in the Presence in My Absence

In the sound installation the Presence in my Absence, I used blue noise as the backdrop in a dark room with five Genelec speakers on stands surrounding the audience. In this installation, audiences were set in total darkness to sense the subtleties in the soundscape. Our ears sense pressure and can detect slight changes (Altman 1992, p. 21). The blue noise surrounding the audience created a consistent noise floor, which the listening body adjusted to. It resembled the rush of air at high pressure, which was most noticeable when momentarily absent in one spot.

I moved my body through the space during the recording process creating sound shadows by momentarily blocking the blue noise between the speaker and the microphone. The sound would periodically drop out of each speaker in the installation, re-enacting the movement and producing the sound shadows. Audience members recalled an initial confusion, questioning their hearing. The sequencing of the sound shadows across the speakers in the room was intended to indicate my moving physical form, similar to what would be felt if my tangible body was moving in the room, passing in front of the speakers and blocking the sound.

The creation of 360-degree sound shadows involved setting up five identical microphones in a circle, surrounding five outward-facing speakers in the centre that were playing blue noise toward the microphones. This is represented below in Fig. 2, with symbols representing the placement of the five speakers in the centre and five microphones on the perimeter of the action. I moved my body around the circle, improvising with pathways and approaching each microphone with the intention of reaching my future audience. Through this process, I both listened for and created the sonic change, responding to and experimenting with the dynamics of different movements. I performed my movements as if the recording equipment were my audience, creating an audible performance for them as they tracked the shadows my body left in the consistent sound. The setup was based on reverse engineering the experience: what acted as playback in the recording process became the audience perspective; the structures recording sound were then producing sound in the final piece.

Fig. 2.
figure 2

Diagram representing the recording configuration for 360-Degree Sound Shadows

Full size image

Rick Altman (1992) examines the language we use to describe variances in sound and highlights the pragmatic but limiting ways we define the existence of sound in the world. The language we use fails to differentiate the quality of sound; we name sound based on its production rather than its perception (p. 19). This limits our evaluation of the sonic experience. It was my impression that audience members had difficulty articulating their experience in the Presence in my Absence. They seemed to lack the words to describe the change in sound, focusing mainly on the potential implications of what had transpired: they had lost hearing in one ear, something had gone wrong in one speaker, or something was blocking the sound. The installation infers the cause of the interference as the explanation for the experienced disturbance. It is the bodily experience of audible interference in sound shadows that affirms my exploration of embodiment as an important element of sonic design.

9 Sound Perceived as a Moving Material Form

As I physically inhabit the recording of sound defined by all the expression and constraints of a human body, I am producing a specific corporeal intention which I hope lands with the listening body and is a more nuanced experience than what could be developed through spatial sound editing.

Sound object theory, put simply, denotes a unit of sound as it is perceived as a whole (Schaeffer et al. 2017, p. 67). Godøy’s (2016) sound-motion objects represent a sound that, as expressed, may be decipherable as a “meaningful” unit of sound related to the movement of its creation. Godøy’s research proposes that “any sound event entails an image and context of a body-motion event” (p. 5). This is important as I position my recorded sound as having perceptible, material form.

The sonomotiongram technique developed by Jensenius and Godøy (2013) investigates the translation of information from movement to sound. Such a cross-modal “translation” is based on the universal nature of shape cognition. This research visualizes similarities between sonified movement, movement performed in response to music, and the characteristics of the music itself (p. 75). The instances where traditional music-making and embodied sonic design overlap allow the listening body to ground within an abstract experience. My embodied binaural spatialization technique, which I will explain in the next section, reproduces sound as a moving material form. The more I abstract the moving body in sound, I will offer recognizable sound actions to help reorient the listening body. In my installation, the third movement, I do this with re-creations of literal sonic events like footsteps, breath, or by tapping a pen while recording. Based on research for the sonomotiongram, I imagine the tap of the pen, as an action sonified, would somewhat resemble its sound data.

I am proposing that in depicting dynamic material form, sound can perceptually reanimate the moving body in space. The aesthetic specificity of movement and sound holds the potential for auditory creations as performance and a felt sense of bodies remanifested in space. Sofia Dahl’s (2022) research on music performance and expression points out that musical cues assist with the ability to track simultaneous action through abstract thinking. This ability to more holistically recognize systems and patterns drives my desire to depict movement using a human body in motion—arguably the most recognizable system of them all. Through my practice-based research, I have made observations about the different modes of understanding that are accessible through an expressive abstraction. The sound object, according to Pierre Schaeffer (2017), is what we hear in the acousmatic performance of sound (the absence of instruments) when we cannot determine the source of the sound (p. 67).

I aim to create the perception of movement in the space around the body and sometimes the sensation of self-motion. Vection is the sensation of motion when no actual motion is occurring. This is what I was trying to achieve in the spinning experiment described earlier in this chapter. Sensing movement when you see the train next to you start to move, but your train is stationary, is an example of vection at play (Clarke 2005, p. 75). Vection is typically experienced visually but can be experienced in listening. Since my early explorations, I have found that to create the sensation of self-motion, the sound source must act like a backdrop or scene, less specific than a point in space and more like the space itself. I have only truly been able to produce the sense of self-motion with the aid of tactile, vibrating transducers in a spatial sound studio, and I attribute much of this to the collaboration of the tactile and auditory senses in the near absence of visual information.

10 Embodied Binaural Spatialization

Sound is choreographed as a moving material form to be sensed by the body in my second technique, embodied binaural spatialization, which functions by recording a single, static sound source through in-ear binaural microphones as I move my body around the sound. As a sonic design technique, embodied binaural spatialization produces sound inherently different—even just slightly—at different points in space. The soundscape, therefore, is composed through movement, using an embodied process with the intention of enabling an embodied experience of the sound for the listening body. This and the sound shadow technique both encourage the listener to perceive sound and space with proprioception (the perceptual sense that situates the body), which in turn activates the body through kinesthetic empathy. This is how I propose to offer the experience of movement without the listening body having to execute it. Imagine being able to feel something without physically experiencing it. In that case, embodiment is a kind of visceral empathy.

Researcher Ana Tajadura-Jiménez is involved in several studies that consider how hearing shapes body representation and posits audition as an important and often-overlooked factor in body awareness and representation (2012). Predictive coding deals with the mental schema of our bodies which is formed by information within our environment and is updated continuously based on new sensory information (Burns 2014). It is with this information that we perceive—and predict—how our bodies interact with the world. One such study found that the sound the impact participants’ feet made on the ground when walking influenced how they felt in their bodies. Participants reported feeling differently about the size of their bodies while hearing the modified sound of their footsteps in action. It is in listening and embodying the sound of my moving body that I propose to offer a new experience of, and for, the listening body.

Embodied binaural spatialization is a way to generate a sense of movement for the listening body. It makes use of a singular perspective, which is first inhabited by the recording subject and then by the listening body, ideally through headphones. This is a process of reversal similar to that of the Presence in my Absence. For example, the listening body experiences a sound in the distance, approaching them from behind, which whips around the right side of their head, circling three times and arching above their head from their right ear to their left before landing in front of their face and fading out. The embodied binaural spatialization process is simply this experience inverted. The sound source—usually a pair of speakers—is a stand-in for the listening body, and I keep it at waist height so that I have room to move easily in any direction. The embodied binaural spatialization technique would use in-ear binaural microphones to create this experience, and I would start walking backward toward the sound source, spinning myself three times clockwise and then bowing my head forward as I lean my right ear in towards the speakers and trace the sound across the crown of my head, into my left ear and turning my face to the sound as I back away from the speakers. Even after creating these movement-scapes, I sense a character in the sound. My body then senses the recorded body as an entity outside of itself and responds to its movement. If I had struggled at a certain point, that tension comes through, and it feels real and still somehow new to me, even moments after living it.

11 The Third Movement: An Installation in the Kinesphere

Fig. 3.
figure 3

The third movement, installed at Emily Carr University of Art + Design, 2021

Full size image

The third movement considers the boundaries of the self, musical expectancy, and the tethers of reality through a track composed over seven separate listening stations. In the installation, pictured above in Fig. 3, the headphones were arranged loosely in the track’s order and hung on plinths of varying dimensions and orientations throughout the installation. The audience was invited to move through the installation and put on the different headphones, each featuring a looped section of the full twelve-minute installation track with a specific condition for listening. At different heights and facings, the length of the headphone cords helped to indicate the intended position and location of the body, while the cushions encouraged the listening body to resign to the floor, as seen in Fig. 4. The audience populated the space, each occupying a specific location in relation to each other while having an individual experience. I edited drones, samples, and voice recordings to move seemingly autonomously in the 360-degree space using embodied binaural spatialization and developing concepts across listening stations. The sound animated the individual kinesphere of the listening body with an allocentric perspective—all while sharing space with other bodies and keeping track of activity around them in the virtual-audible and tangible space. The installation created a situation that was constantly permeated.

Fig. 4.
figure 4

Audience member lays down to experience the third movement sound installation at Emily Carr University of Art + Design, 2021

Full size image

While documenting All Bodies Dance Project (ABDP)’s work It’s Enough (For a Rooftop), taking place on the rooftop parkade of the Sun-Wah building in Vancouver, I experienced a new sonic illusion. One that re-created a site-specific, past sonic event and inspired a more narrative feature of the third movement. I was reviewing an ambisonic (360-degree) recording onsite using a pair of headphones while crouching next to the microphone’s recording position as dancers moved to the shade for a break. I started playback, and the voice of the rehearsal director started up behind me, asking the dancers to take their positions. I looked behind me and found the entire group was still quietly occupying the shaded end of the rooftop parkade, including the rehearsal director. I was struck by the distinct sensation of someone behind me, her presence commanding action. Played back, the ambisonic recording convincingly re-created this sonic event because it was being reproduced at the exact site of its recording and occurring as if at the same angle behind me. In the spinning experiment from earlier in this chapter, I noted how the sound of the music in the space helped promote the sense of self-motion based on convolution resonance. The same effect here makes listening to a past sonic event—at the same site of its recording—a more convincing re-enactment of the event. That day, on the rooftop parkade of the Sun-Wah building, the sound was so realistic that I was physically responding to this information before I could fully make sense of it.

The recording I created for the first set of headphones featured one of these past sonic events, created and recorded in the exact same playback position for the listener. I walked up to the microphone slowly, making sure that each step I made was clear and audible. At this point, the hyper-reflective process was heightening my senses. I spoke into the mic intermittently, saying things like “hey” as if trying to get someone’s attention. I said, “it’s okay, it’s supposed to do that,” in response to electronic glitch sounds that I later layered onto the track to give the illusion of a bad wire connection. In a hyper-reflection process, the recorded action was layered to manipulate and redistribute the experience of liveness. The sensory confirmation from the environment within which we perceive our bodies generates this sonic illusion using predictive coding. Retaining the specific audible features of the building and installation space made plausible the sound of footsteps and the voice coming from behind the listening body.

The sensory anticipation and deception presented this work as playful trickery. I addressed the space between people—which was no more than a few feet, as pictured in Fig. 5—by occupying the perceived auditory-kinesthetic space with sound, which the listening body entered through headphones. The headphone cord represented a tethering to the site of the present experience, while musical development, dialogue, perceptible movement, and the site-specific past sonic events projected another reality. Portraying sound as a moving material form, the third movement brought attention to the conditions of certainty. I wanted the listening body to question what they were hearing and to feel at odds with what they were seeing and what they were sensing.

Fig. 5.
figure 5

Audience members populating the third movement sound installation, Emily Carr University of Art + Design, 2021

Full size image

12 Technology and Access

It is important to note that my work is not generated by technology but mediated through sound equipment and software. The recording process is essential in communicating nuanced work that could only be produced by a body. Presenting my work using audio technology is the point where the research is embodied by others. One benefit to recognizing the impact that daily media has on the human psyche is that it indicates a unique permeability through certain means. I aim to draw up bodily knowledge rather than implanting a message of my own. The technologies I work with are connective, they act as a means of extending my corporeal expression and enable communication from one body to another. Working this way becomes an ever-changing mode of creation through which to conceive novel experiences.

The language we use to understand the world is largely spatial; in fact, a good portion of how we communicate depends on orienting concepts and relating them spatially within our lives, like “diving into a subject” or saying something is “in the realm of possibilities” (Tversky 2019). This is how we qualify and understand the significance of events and circumstances.

Singer-songwriter Imogen Heap is working with a team of engineers, artists, and designers on the creation of gloves that use the artist’s hand movements for the live performance of electronic music (Nosowitz 2014). The Mi.Mu Glove is a set of gloves comprised of specifically placed sensors that act like a synthesizer. Although the glove is meant to enhance stage presence, it allows the artist to approach the creation of each sound differently, thinking spatially and through the body (2014, para. 4). Based on the theory of multiple intelligences, workflows that enable different ways of understanding sound creation are generative as well as innovative. I am using embodied processes for creation where movement functions as sonic design, doing much of what could be done with software through the moving body. Working this way enables greater access to this type of creation for artists without the tools or education to produce such soundscapes.

Artist jamilah malika abu bakare’s interdisciplinary practice prioritizes listening over looking as she seeks justice through art-making (2021). Abu bakare speaks of the gate-keeping of technology not only by gender, race, and class but of the developing art forms themselves, indicating that the expectation for high-level production narrows the scope of voices represented. She emphasizes the power of “art that makes you want to make art,” which encourages engagement and inquiry as opposed to upholding an artistic hierarchy (2021, Sound as freedom [Artist talk]). To abu bakare, art that brings about change is less about merit and more about continuing the conversation. It is the inclusivity of her practice, which involves anti-oppression as well as an emerging artist’s initiative, that truly beckons other makers. Her provisional use of technology, purely for recording and playback, places more importance on the act of listening than demonstrating mastery of technique. Abu bakare goes as far as to say that the low-tech appeal of her sound and video works underlines the priorities of her practice, demonstrating a refusal of the commodification of media practices.

13 Where Do We Go from Hear?

Abstract art facilitates a heightened awareness that makes sensation more available. Audio, like video, can offer an experience that we may never be able to enact physically by envisioning it, or in the case of sound: by imagining it. Physical embodiment is not reserved for the moving, able body. This point most demonstrates my enthusiasm for creating work for the senses: perception of sensation and actual physical sensation can have the same impact on the body.

It is the availability of research on sound and movement that initiated a line of inquiry on the effect of sound on the body, leading to the development of my artistic research. Ultimately, I am exemplifying an internal, felt experience. This is in an effort not only to be witnessed but to share something as it is felt. Sound installations created through embodied binaural spatialization and sound shadows could one day be reproduced as dance, with intricate movement patterns mapped onto different surfaces, apparatuses, and spatial schemes. This would require more sophisticated technologies, but I am interested in the idea of how much movement the listening body can track simultaneously and at what point the motion amalgamates, amounting potentially to an entirely different form. Can movement be heard in the way a harmony of tones becomes a chord, as a chorus of action?

As I work with embodied sonic design, I challenge the approach that defines music so that the definition and application of sound might expand. The more experimentally I work, the more intently I listen and the more I appreciate what I can hear. I am working in this way to deepen the experience through embodiment. Appealing to the senses is my way of offering an experience that is beyond visual representation, one that is individual but shared. In creating awareness through physical attunement, I endeavour to regulate systems out of balance—the systems within us and those that govern our bodies. Embodied sonic design provides access to alternative states through embodiment: shifting how you feel in a particular setting. Inhabiting the body in a new way can empower us to live differently within conditions we cannot control. Acting as an empathetic intermediary, I create shareable spaces using embodied sonic design, so we can understand beyond our own circumstances. A focus on other ways of knowing, through embodiment, is simply my way of bringing balance to a world that I believe is suffering from its reliance on visual assurance. A sensorium is an escape from rules we did not make yet must adhere to. I offer my creations as a refuge where feelings and ideas can take shape.