Keywords

1 Introduction: Corals Are in Crisis

Many coral species cannot adapt quickly enough to warming ocean temperatures and increasing acidification, threatening reefs worldwide and creating the potential for a catastrophic loss. The habitat created by coral reefs makes them fundamental to the sustainable diversity of ocean fauna, and to all living things—including humans—that rely upon this part of the earth’s ecology.

Our audiovisual installation regards coral reefs as the ‘voice’ of the Anthropocene, hence the title Choral. Reefs play a foundational role in providing habitat for a quarter of all marine species and these ecosystems are in crisis. Although they face challenges from multiple directions, we are approaching our piece from an optimistic perspective. Techniques for fostering recovery and regeneration are already under development: a dedicated part of the scientific community is determined to find ways to make these methods scalable, and this is cause for hope.

Through our creative work, we endeavor to join this collective effort by contributing to a larger-scale human response. Although scientific research informs our fundamental approach to the visual appearance of various coral species, the way corals behave in the work reflects our shared imagination through speculative underwater world-building—rather than emulating a literal documentary. We were invited in Spring 2017 by the curator for 150 Media Stream [1], Yuge Zhou [2], to make a moving image artwork for 150’s unique wave-shaped 89 vertical panel LED-screen lobby display. This custom-designed technology allows for working at a large scale—the Media Stream display measures 6.5 by 47 m—while the nature of this venue in the heart of downtown Chicago allows for a larger audience to encounter and consider this topic. Our response, Choral, is a computer-generated 3D animation with electronic sound that addresses our inquiry focused on the ecosystem of coral reefs (Fig. 11.1).

Fig. 11.1
figure 1

Photo Michael Salisbury [3], image copyright ©Michael Salisbury and reproduced by permission

Choral at 150 N Riverside Plaza, Chicago Illinois, USA.

Full size image

2 Initial Research Phase

Having attended Ars Electronica for five years, three Documentas, several Venice Biennales, as well as having participated in the STRP Festival, the National Art Museum of China’s TransLife Triennial, the Taipei Digital Art Festival, and encountering a wide range of art installations incorporating sound, moving image resources, or combinations of the two, in addition to maintaining a collaborative praxis as artists, both of us are professors who have taught numerous classes since 2003 that involve new media and installations. We take our responsibilities to stay well informed seriously as it is integral to our pedagogical activities. The following essay may best be understood as intentionally circumscribed in its scope, since a more expansive survey of digital installations as a genre would exceed our remit.

As artists we have been fortunate to have a range of public platforms available to reach a diverse audience. The concern for what we, as a species, have done to the planet is reflected in our work as individual pieces are used as opportunities to probe deeper into these topics while assessing the effectiveness of different modes of expression on audience engagement. Given our history of collaborating on projects—pr!ck (2006–08) [4], scale (2009–10) [5], Bird (2011–14) [6] —that involved aspects of what two physicist colleagues have termed “cultural processing” [7], we felt it necessary to delve into more informed research beyond our initial encounters with journalistic reports dating from 2014 regarding the catastrophic events that coral reefs were suffering around the planet. Determined to gain a better grasp of the situation, we first undertook a program of self-education through a variety of documentary films and videos, paralleled by perusing a selection of more recent scientific papers.

In the course of learning more about the multiple causes and disastrous consequences of coral bleaching and death, we encountered the research program of two Chicago-based scientists, Luisa Marcelino (Research Assistant Professor of Civil and Environmental Engineering, Northwestern University) [8] and Timothy Swain (Postdoctoral Fellow, Northwestern University and the Field Museum) [9], whose published articles were a source of inspiration to us, and who agreed to meet us in person. Prior to our March 2018 meeting, we viewed a WTTW television segment where both were interviewed and we were struck by Dr. Swain’s observation, “The coral reef ecosystem may be the first ecosystem lost to climate change” [10]. Though the topic was initially grim, our discussion with Marcelino and Swain was informative and productive. An afternoon conversation with these scientists absolutely validated our concerns regarding the critical condition of the global ecosystem of the reef and our commitment to developing the project for 150 Media Stream. We learned that the Marcelino Research Group’s work at Northwestern points to coral polyps as architects, wherein the fractal properties of their skeletons redistribute light as a photosynthetic resource within the colony. Further study of this phenomenon suggests a relationship between faster-growing species and increased vulnerability to bleaching.

At the end of our meeting, Prof. Marcelino offered to curate a number of critical papers for us to study and Dr. Swain invited us to join him at an upcoming visit to Chicago’s Shedd Aquarium to observe some of the coral works that he was engaged with in their laboratory. Having a chance to see these animals close-up and in-person brought a more immediate level of appreciation of their charismatic qualities. It was moving to see how tiny these creatures were while playing such a large role in marine ecosystem sustainability and by extension human economic dependency on that very ecosystem. As Rebecca Albright says:

Although reefs cover just 0.1 percent of the ocean floor, they support nearly 25 percent of all marine species, including fisheries that feed millions of people worldwide. They also provide natural breakwaters that protect coastal communities by reducing wave energy by up to 97 percent and wave height by up to 84 percent. And they generate vast tourism revenue. If we lose reefs, we jeopardize the livelihoods of 500 million people and more than $30 billion annually in goods and services [11].

Many people benefit unknowingly from the coral reef habitat; numerous articles refer to the reefs as the “rainforests of the sea” [12] or “the medicine cabinets of the 21st century” [13], from whence a number of anticancer, antiviral, and pain management drugs are being explored, as well as bone graft material. During our behind-the-scenes encounter at the Shedd Aquarium, one of the staff members remarked that although they had valuable and instructive coral displays, they felt they weren’t reaching enough of a diverse public to get their message across, and Swain replied: “That’s what Marlena and Jay do, as artists”. Consequently, we realized Choral could play a role beyond expressing our personal commitment to Anthropocene issues; it might also have the potential to raise public awareness and connect that awareness to the marine biology community that is actively working on solutions to the problems. Helping this community become more visible might ultimately translate into the public policy decisions we collectively need to be enacted for corals to be saved. One of the papers Marcelino referred us to was “Coral Reefs in the Anthropocene” [14]. Among the noteworthy points found:

“[…] research has revealed provocative complexities in the expanding knowledge base about corals and their ecosystems, thus stimulating our imaginations regarding alternate ways to envision the future of reefs” and “Increasingly, coral reef scientists and managers encounter previously unseen configurations of species” [14]

According to Terry Hughes, the reality is that:

reef ecosystems are more dynamic and patchier, as well as increasingly different to anything that people have encountered before. Embracing this paradigm shift will necessitate a transformation in the governance and management of these high-diversity ecosystems [14].

There won’t be any single answer to these problems; solutions to fostering sustainability for corals in the future could build upon a wide range of methods developed by Marcelino’s group as well as many other scientists working in the field. Some of these include the creation of a meta-level tool to correlate bleaching data from all previous coral studies into a coherent worldwide index [15] and identifying species better adapted to survive. We learned from Albright’s article that:

Mary Hagedorn of the Smithsonian Conservation Biology Institute has established the first genome repository for endangered coral species. […] Her team has developed a freezing system for sperm that can be applied to a wide range of coral species. To date, the team has successfully banked 16 species from around the world (2 percent of the earth’s estimated 800 species) [11].

Our optimism increased when we learned about the work of the late Dr. Ruth Gates at the Hawaii Institute of Marine Biology and her collaboration with Madeleine van Oppen of the Australian Institute of Marine Science to develop next-generation supercorals. Their labs are working on selective breeding and epigenetic tuning to tolerate warmer sea temperatures, and selective breeding of the symbiotic zooxanthellae and the bacteria that comprise the microbiome of coral polyps to breed more robust colonies. SECORE International is working on both increasing genetic diversity and fostering the odds in favor of coral larvae surviving past their initial vulnerability to being eaten by other marine creatures; when they are big enough to be outplanted, these new generations of young corals can help repopulate damaged reefs. One of the best designs for an economically scalable substrate for baby corals to grow upon is a grapefruit-sized concrete tetrapod, textured so that the larvae can readily attach themselves and start their reef-building. These tetrapods are featured in one sequence of Choral as a symbolic representation of the collective efforts of humans to channel their ingenuity toward finding a solution to the bleaching and reef degradation crisis. (One of our contacts at the Shedd Aquarium, Mark Schick is a collaborator with SECORE in the tetrapod project.) Other, more speculative approaches under development include deploying groups of underwater robots that would re-cement broken coral fragments to the deep-sea cold-water reefs they came from, aiding re-growth.

Having first-hand discussions with scientists who are dedicated to identifying and addressing the complex issues that promote coral bleaching was encouraging. These experiences bolstered our decision to create an audiovisual artwork that engendered hope and fostered public engagement rather than despair and cynicism. A meeting with Hobson [16] who is a researcher at NORC was also valuable to our project; drawing on personal experiences from research projects undertaken when she was based in Australia, Hobson told us that “a noisy reef is a healthy reef” which inspired us to investigate this aspect and subsequently became a factor in shaping the conceptual approach to sound design and soundtrack orchestration in Choral.

3 Project Development, Phase Two

A steady intake of information fostered the necessary processes of refinement and metamorphosis, guiding the project’s vector away from didacticism toward a more poetic direction. A project of this scale compelled us to undertake two different, though complementary research tracks. The scientific data obviously had to be assimilated and understood to inform our conceptualization of the project, but we also needed to comprehend what kinds of hardware, software, and personnel resources were needed to realize our vision, and to develop a budget and a team that could feasibly support that goal. With the scientific side in full swing by the onset of winter 2017, we accepted an invitation to participate in the Species/Biodiversity Loss panel of the conference Why Do Animal Studies?: The Turn to the Quasi-, Post-, Anti-, Non-, Para- [17] in April 2018, with our presentation “Choral, a work in progress”. At minimum, it was a chance to get expert feedback from scholars whose critical thinking and writing on non-human others and Anthropocene-centric issues had been impactful on our own mode of inquiry and efforts. As is common in these types of conferences, it was also a remarkable opportunity to be immersed in the multidisciplinary confluence of scholarly and artistic projects that comprise the field.

One of the strengths of how localStyle’s collaborative approach has deepened since its founding in 2000 is the way we are able to pursue independent development tracks in our working methods based on our individual skill sets and areas of expertise, while maintaining a constant conceptually driven dialogue that revolves around the central thrust of any specific project. Inspired by the hospitality of corals to create habitats for other creatures, Novak began modeling, texturing, and animating individual corals—devoting time to both individual colonies from various species as well as to detailed hexacoral polyps—and to assemble a synthesized version of their biome utilizing 3D software tools (Autodesk Maya, Arnold Renderer, Pixologic zBrush) on a Mac Pro workstation. In the conference, her slides incorporated digitally created images that symbolically represented coral bleaching and the degradation of the ecosystem (Figs. 11.2 and 11.3). Based on audience feedback, this imagery turned out to be effective for attendees in the academic context of various presentations zeroing in on challenges that ranged from justice, law, and ethics to postcolonialism to disruption on a variety of planetary scales. These preliminary images thus played a crucial role in our conceptual development of Choral, even though we ultimately decided to tactically focus the installation’s mode of address toward one that emphasized the hopeful side of human efforts to rescue coral. As with any environmental or climate-related issue, the presence of the political dimension is never far from the surface, and the operational question arises as to how people can be motivated to engage and empathize with a topic so as to enable policy-making to occur in positive ways.

Fig. 11.2
figure 2

3D image: Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Study for Choral: Reef Ecosystem Degradation.

Full size image
Fig. 11.3
figure 3

3D image: Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission.

Study for Choral: Zooxanthellae expulsion.

Full size image

Shortly after meeting with the Northwestern University scientists, the project implementation side of Choral began to accelerate. The organization that developed for the installation at 150 N Riverside Plaza placed Novak as co-producer, director, project manager, lead 3D visual artist (concept, modeling, texturing, and animation), and motion graphics conceptual developer; Yim was the co-producer and composer/sound designer as well as being involved with the 3D and motion graphics conceptual development. Clearly a work of this scope would have much to gain from having project assistants; they would benefit from the experience, contribute to the work in a multitude of ways, and be credited for their roles in a high-profile project.

In Spring and Fall 2018, Novak assembled a group of young artists who had formerly studied with her at the School of the Art Institute of Chicago. At an early stage, she compiled a collection of references as an encapsulated database to share with team members. These comprised science-based pieces of journalism in print, web browser, and video formats, organized into three main groups:

  1. (1)

    how corals live,

  2. (2)

    reasons for humans to care about corals, and

  3. (3)

    factors threatening their existence.

The first group of references concerned coral taxonomy, structure, anatomy, habitat, behavior, etcetera. It was important for everyone on board to assimilate some of the basic information about these animals: coral polyps have a symbiotic relationship with photosynthetic dinoflagellates; their survival depends on these algae that provide between 90 and 98% of their nourishment; these zooxanthellae live under the epidermis of each polyp; the polyp’s epidermis is coated with bacteria; and these three organisms appear to benefit one another. Especially pertinent were key points provided by the Coral Reef Alliance:

Coral reefs are often referred to as the medicine chests of the sea. A number of creatures found on reefs produce chemical compounds that have been isolated for human applications—and many more are yet to be discovered. Scientists have developed treatments for cardiovascular diseases, ulcers, leukemia, lymphoma, and skin cancer, all from chemicals in reef plants and animals. Other compounds reduce inflammation, kill viruses, and relax muscles. The beautiful and fragile creatures of our coral reefs have the potential to make even greater contributions to our lives by providing new cures for life-threatening diseases. More than half of all new cancer drug research focuses on marine organisms. In addition, coral’s unique skeletal structure has been used to make our most advanced forms of bone-grafting materials [18].

In addition to the stresses of temperature-induced bleaching and ocean acidification weakening their skeletal structures, other factors (many of which are under direct human control) combine to threaten coral’s existence: in the natural competition between coral and seaweed, warmer water combined with fertilizer run-off gives seaweed and other macro-algae the advantage; overfishing of species that ordinarily keep seaweed in check by feeding on it exacerbates the stress; governmental policies that revoke environmental protections for existing marine national monuments result in more pollution from industrial exploitation such as mining and oil development; even tourism can be a factor as products like many sunscreens are toxic when introduced to the water. (The most recent evidence points to these products also being toxic to humans.)

As work progressed, we held group screenings of selected documentaries to harmonize our perspectives. Joining localStyle in preproduction concept development were Sally Jo [19] and Malu Ayers [20], and project intern Shinuo Snow Xu [21]. As team assignments solidified, Ayers and Xu worked as 3D artists (modeling, texturing, lighting, animation); Max Crider [22] built and animated 3D rigs for polyp and coral animation in the close-up scenes. Nathaniel Gillette [23] joined us in the Fall as a 3D and 2D artist and became motion graphics coordinator. Also in the Fall, Mak Hepler-Gonzalez [24] joined the Choral team as assistant producer, rendering manager and 3D artist involved in all aspects of 3D production, along with post-production, conceptual, and technical development; he contributed several advanced 3D techniques for compositional layout, rendering, and animation that would serve the project until its completion (Fig. 11.4).

Fig. 11.4
figure 4

Photo Jay Alan Yim, image copyright ©J. A. Yim 2019 and reproduced by permission

Choral team members working at Unspecified Research Lab (top to bottom: Gillette, Novak, Hepler-Gonzalez, Ayers, Xu)

Full size image

4 Workflow

Due to having an artist’s residency abroad during some of this period, we set up a shared online project folder and a blog to stay involved with each other’s progress; we also scheduled regular Skype meetings while away. Once back in Chicago, studio sessions resumed. Our working method evolved to include collaboration on many levels: in some cases, a single artist modeled, textured, lit, and animated their models; at other times, they would hand off models they made for another team member to texture, while another artist did the lighting and another executed the animation. If one of us developed a new material that would be suitable for others to use, or a lighting-set, or an animation technique that would be effective for each other’s models, we shared these 3D elements. For the most part, our roles were fluid, unlike production hierarchies where a single person works on modeling and another solely engages with texturing, and so forth. Except for one artist who was only involved with much of the rigging, we exchanged and reversed the majority of our roles. This created a special sense of team effort and connection to the final piece. When one of our valued members relocated to another part of the country, we continued to work together via retrieval and development of their uploaded files. Choral would not exist in its current state if not for the synergy of this solid collaboration.

We were fortunate to receive supplementary grant funding from the Illinois Arts Council and a Faculty Enrichment Grant (SAIC) that supported aspects of the project; several new computers were necessary to address the rendering time needed for the unusually high resolution of each frame of animation. In addition to the previously mentioned software programs (Maya, Arnold, zBrush), Allegorithmic Substance Painter was used for texturing and Adobe After Effects was used for compositing. Over the course of the project, two custom-configured PC workstations were acquired by Unspecified Research Lab, which is localStyle’s studio facility. Sound design/scoring took place on two Mac workstations. Throughout the project, as many as eight laptops were used in conjunction with the desktop machines. Work continued in a mixed Mac/Windows environment, tailored to take advantage of the capabilities of each platform (Fig. 11.5).

Fig. 11.5
figure 5

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Choral team members working at Unspecified Research Lab (left to right: Yim, Ayers, Hepler-Gonzalez, Gillette, Crider).

Full size image

Fortunately, we had designed our studio space to be flexible in usage, which allowed a number of artists to work together comfortably and in close proximity to the kitchen and seating areas where we could take breaks as necessary. We regularly provided lunches and dinners, and team members ate together depending on the work periods (Fig. 11.6). Sharing home-cooked meals, often from items grown in our garden, built a closer sense of community; several times an artist from the team would also contribute cookies or bread they had baked. Group meal conversations sometimes addressed recent project achievements or trouble-shooting concerns, but mainly it was a time to focus on positive topics.

Fig. 11.6
figure 6

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Choral team members enjoying a lunch break (clockwise from upper right: Yim, Gillette, Xu, Hepler-Gonzalez, Ayers).

Full size image

5 Compositional Design and Technical Challenges

There were a number of design challenges unique to this project. One of them entailed the sheer physical size of the video display; with the tallest panels standing at nearly 22 feet and the full array spanning 153 feet in width, we needed to make the visual elements narratively coherent even though it is impossible to stand back far enough in the lobby to see all of the LED blades simultaneously from the center. It is quite a different experience to visually compose a scene with digital models being made, textured, and animated on 28-in. computer screens having 3840 × 2160 resolution; we could assess the design in its entirety. Though we are fortunate to have a 4.9 m (diagonal) projection screen in-house at URL, that is still dwarfed by the size of the tallest blades (6.54 × 0.225 m) at 150 Media Stream. The physical scale impacted team decisions regarding formal qualities such as repetition, proximity, motion, and pacing, requiring care as to where visual elements would be located to create the most dynamic mise-en-scène. Strategically, the array’s width helped direct the temporal proportions of different passages in Choral, as the situation compelled the viewer to move actively through the space to take in as many CGI corals as possible.

The second major factor in the design was working with the negative space that is an integral component of the display: we had to be conscious of this site-specific parameter to ensure that animated elements retained maximal legibility as they negotiated the voids between panels. Though it was quite useful to have a TIFF mask composited as a layer to preview image sequences on our computers, we appreciated the difference it made to see and hear the draft versions of Choral in situ. In a sense, it was visually somewhat like working with a reverse zoetrope in that the alternating slits and opacities worked together to create an interrupted continuity.

LED display resolution was a corollary aspect of the size factor: 150 Media Stream’s total pixel count was 15360 × 2160, the equivalent of four 4K screens arranged in a horizontal panoramic view. Rendering moving image sequences at high resolutions is intensive in terms of both time and computational resources. We had non-exclusive access to a multiblade render farm, and non-exclusive access to several banks of computers, in addition to the machines at URL. Emily Kuehn [25]—who is an exceptionally resourceful staff colleague at SAIC—provided crucial support and assistance in addressing the unique and precise Maya template for the 89 screens (which measure a variety of heights and widths), and optimizing the render sequencing setup for the 16K display.

The acoustics of the building conditioned the approach taken to orchestrating the electronic soundtrack and equalizing the mix: the presence of many hard, reflective surfaces—stone, glass, metal—and the height of the atrium effectively disperses much of the audio signal and makes it nonlocalizable. Architectural design at 150 N Riverside Plaza produces acoustic results that are comparable in some significant ways with the reverberant spaces often associated with the performance of large choral works. This made the decision to base the soundtrack on Thomas Tallis’ forty voice motet [26] an apposite one, both for these acoustic characteristics and because the eight-channel audio system corresponded to the polychoral division of voices in his score into eight separate vocal quintets. Additional motivations for choosing this point of departure will be discussed below.

6 Choral Animation Summary

The introduction begins with waves on a dark sea (where the ocean is a digital simulation). In the tradition of an origin myth, we see friendly bioluminescent polyps ascend to take their places amidst a field of stars in a night sky filled with simulated constellations (Fig. 11.7).

Fig. 11.7
figure 7

Photo Michael Salisbury, image copyright ©Michael Salisbury and reproduced by permission

Polyps rising.

Full size image

Stylized bubbles rise against the water to become stars; many are replaced by the astral polyps. As the polyps wink out, the celestial field metamorphoses into a digital wire frame that descends to form the contours of a seabed, gradually evolving into more solid terrain (Fig. 11.8).

Fig. 11.8
figure 8

Photo credit: Michael Salisbury, image copyright ©Michael Salisbury and reproduced by permission

Celestial polyps

Full size image

As an aqueous curtain rises, a richly populated reef full of many species of coral is revealed, and the virtual camera pans at a deliberate pace across the ocean floor (Fig. 11.9).

Fig. 11.9
figure 9

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Seabed revealed.

Full size image

Being digitally modeled, our reef places all of the corals front and center to emphasize their importance to the community of fish, other creatures, and marine plants that would comprise and inhabit this ecosystem in nature. Throughout our 3D animation, the corals are singing when visible (Fig. 11.10).

Fig. 11.10
figure 10

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Coral reef

Full size image

The digital camera zooms in and slowly circles around several of the more charismatic colonies on this imaginary reef. Despite most coral polyps being individually very small animals, in the aggregate, they play an outsized role in the formation of reef ecosystems, which is our motivation for modeling them in detail for these close-up views (Figs. 11.11 and 11.12).

Fig. 11.11
figure 11

Photo Jay Alan Yim, image copyright ©J. A. Yim 2019 and reproduced by permission

Coral close-up view.

Full size image
Fig. 11.12
figure 12

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Active polyp close-up.

Full size image

The intended goal of these two sequences is to engage audience members with the awe-inspiring beauty of coral: to reconnect them to personal memories of experiencing coral in the sea or other environments. Hearing viewers remark, “We should go back to the Shedd Aquarium” after seeing our piece in the lobby of 150 N Riverside underscored the importance of making this kind of contact since Chicago’s Shedd Aquarium has very good coral exhibitions with detailed information regarding the stresses on coral ecosystems and how those risks will impact our lives if corals are allowed to perish.

The following passage features many computer-generated tetrapods with growing baby corals descending through the water (Figs. 11.13 and 11.14). These represent the small concrete forms that researchers have designed to serve as ideal substrates for newly attached larvae to grow upon. Outplanting tetrapods is one of several complementary solutions currently under development and the Shedd is involved in promoting this approach. As artists, we had hoped that an engaged public would be curious to know more about what the tetrapods are and how they relate to coral reefs; during the test screenings, we have been pleased that this sequence has sparked people’s interest and initiated conversation.

Fig. 11.13
figure 13

Photo Jay Alan Yim, image copyright ©J. A. Yim 2019 and reproduced by permission

Tetrapod with growing coral close-up.

Full size image
Fig. 11.14
figure 14

Photo Michael Salisbury, image copyright ©Michael Salisbury and reproduced by permission

Tetrapods descending to ocean floor.

Full size image

Next, the scene crossfades from objective potential to the metaphysical, as the corals in Choral dance and demonstrate choreographic prowess and graceful coordination. They use their bioluminescent qualities in place of costumes. The dance aspires to usher in their miraculous spawning event (Figs. 11.15 and 11.16).

Fig. 11.15
figure 15

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Dance sequence close-up.

Full size image
Fig. 11.16
figure 16

Photo Jay Alan Yim, image copyright ©J. A. Yim 2019 and reproduced by permission

Dance sequence, long view.

Full size image

Finally, the magical moment arrives that happens once per year when coral colonies spawn in synchronization with other colonies of the same species: eggs and sperm are released by the millions to join in the water and produce new coral larvae. For those species whose preference is to incubate their larvae inside the bodies of their polyps, the mother polyps shelter their eggs and are fertilized by sperm from male polyps; when the time comes, the larvae emerge from the mouths of their mothers. Regardless of the method, the new corals and gametes rise in the water column toward the starry sky, swimming up to the surface as they are attracted to light, and eventually falling back to the seabed where they will begin new colonies. Symbolizing the cycle of regeneration, our animation is formulated to seamlessly loop from the spawning sequence back to the ocean scene at the beginning (Figs. 11.17 and 11.18).

Fig. 11.17
figure 17

Photo Marlena Novak, image copyright ©M. Novak 2019 and reproduced by permission

Spawning coral sequence close-up.

Full size image
Fig. 11.18
figure 18

Photo Jay Alan Yim, image copyright ©J. A. Yim 2019 and reproduced by permission

Spawning coral sequence long view.

Full size image

7 Sound Design and Electronic Score Summary

A healthy reef is in reality a noisy place, thus, our narrative begins with a rhythmicized evocation of that percussive sound world, gradually superseded by more sustained and contemplative textures. When the seabed materializes and we see the reef replete with coral colonies of many kinds, we hear the polyps singing polyphonically; it is left to the viewer whether one chooses to interpret this as metaphor or fantasy: that corals are the voice of the Anthropocene.

These non-human choristers are constantly present in the sound design of Choral until the final gamete sequence fades back into the ocean scene of the opening frames. Conceptually, the soundtrack began as a recomposition project, taking one of the most celebrated choral works in the early music repertoire, Thomas Tallis’ 1570s motet for forty voices “Spem in alium”, and applying an idiosyncratic version of musica ficta [27] so pervasively as to relocate the harmonic world of the motet to that of the present day. (At this point, it may be worth noting that this author first encountered the Tallis score as an undergraduate composition student in 1977. It was immediately elevated to my personal list of desert-island discs, and from the time of initially becoming a professor of composition, has for decades since then been recommended as a study score to my own students. It is truly a marvel of counterpoint and spatialisation, arguably superior—particularly in terms of melodic independence—to Alessandro Striggio’s [28] own forty voice motet, “Ecce beatam lucem”, which is typically seen as the precursor to and inspiration for the Tallis.) The timbral world of the music is similarly updated to incorporate otherworldly vocal sounds with electronic textures far removed from the sensibilities of the Renaissance. In the spirit of technical inversion, all of the percussive sounds are triggered by the rhythms of Tallis’ motet, whereas the smooth resonances that create the impression of being in the ocean have been fashioned from underwater hydrophone recordings of reefs, which are full of clicking sounds with short attack and decay times.

A variety of software tools contributed to the workflow: Finale for MIDI score input and recomposition in a staff notation environment (due to my classical compositional training, staff notation offers more effective control over counterpoint and harmony than the piano roll display in Live); Ableton Live as a MIDI track host for softsynth instrument playback and recording (including Native Instruments’ Absynth 5); Sound Forge for time compression and as a plugin host for DSP and sound design (transforming the hydrophone recordings, shaping samples for the descending sounds in the tetrapod scene, spectral blurring of the vocal textures); Audacity for timestretching; and ProTools for overall track assembly, synchronization with video, and mixing. In the plugin department, Eventide’s Blackhole reverb and Unfiltered Audio’s BYOME were invaluable.

The first (and most intensive) step was undertaking recomposition of Tallis’ motet in Finale. Working in score view gave me the greatest degree of control as chromatic modifications—with increasing deviations—were made to the original score. Garritan Orchestra samples were used to record the basic choral textures, but these were modified substantially in Sound Forge with granular spectral tools and other audio processors. The MIDI data from the Finale file was exported to Live for further development. In Live, the combined MIDI data for all forty voices of the complete motet were used as a single clip to trigger a variety of softsynth instruments; each was recorded and rendered as an 8.5 minute audio track. These tracks were synchronized to the same start time in ProTools because the scoring strategy was somewhat unconventional: (1) to have the recomposed motet play uninterruptedly from beginning to end, thus ensuring continuity, while (2) the electronic orchestration changed for each animation sequence in order to best support the scene, via automation curves when making the final mix. MIDI data was processed using some of the commonly available utilities found in most modern DAWs: automation of pitch bend data made the microtonal retuning of the harmonic series arpeggios in the final scene easily achievable and straightforward, and a moderately complex deployment of an arpeggiator function in the techno-prog scene was used to create the swirling melodic arabesques.

8 Conclusions

Exploring this topic through the use of computer-generated motion graphics and 3D software has created an opportunity for us, along with our artist team members, to look very closely at these remarkable animals in order to observe their attributes as best as possible. But it didn’t stop at digital mimicry; using these tools compelled a type of looking that resulted in awestruck wonder at the complexity and beauty of coral’s functions. The privilege of having this enormous, custom-designed ultra-high resolution LED display to present our project in a public space allows a wide audience of diverse means and pursuits to engage with this subject. We hope that some portion of the viewers will be motivated to consider what role they may themselves play in fostering the sustainability of coral reefs, as scientists in the field have made it clear that we must act now without hesitation to save this ecosystem from extinction.