Keywords

Twittering Timbrados

On January 23, 2015, a Maastricht University lecture hall featured four Timbrado canaries, two Edison phonographs , a vintage gramophone , a serinette , a piccolo player, an artist-researcher, and an audience in eager anticipation. The artist-researcher, Aleks Kolkowski , intended to reenact and demonstrate how bird sound was recorded by bird researchers and the phonograph industry in the early years of the twentieth century. It was one of the events we organized in the context of our Sonic Science Festival , an outreach activity accompanying the Sonic Skills research project.

Much went “wrong” during the demonstration . It was not that the canaries did not make themselves heard. We had expected them to remain silent , as canaries normally do not sing in January. With this in mind, we had brought along a serinette , a mini-organ originally employed by eighteenth- and nineteenth-century bourgeois bird lovers to seduce or instruct birds to sing. 1 But our Timbrados were trained birds and were used to twittering under the most stressful conditions, such as the contests their owners sent them to. They responded to the slightest high-pitched sounds, and burst out in loud concert when one of us played the serinette .

Recording their sounds with a mechanical, early twentieth-century phonograph —without the use of microphones —turned out to be a much harder nut to crack. In addition to the phonographs we had available, Kolkowski had brought a portable oven to soften the wax on a cylinder , two different needles (one for recording and one for replaying), and a horn to capture the sound produced by the birds . That sound’s acoustic energy had to make a membrane vibrate and move the needle, which would leave its traces in the wax . Kolkowski shouted the date and place of the event into the horn , as early twentieth recordists would have done. Unfortunately, when the cylinder was replayed, his voice sounded too high and his words too fast—he had cranked up the phonograph too hard, making the cylinder run too fast. By explicating this, Kolkowski demonstrated the had listened diagnostically in order to understand the disappointing quality of the recording.

Even worse, the cylinder replayed the bird sounds only faintly, if not at the very threshold of hearing . Apparently, the sound waves had not reached the membrane with sufficient energy. Nor did Kolkowski know for certain whether the wax had been heated to the right temperature. So whereas the audience heard the birds sing quite loudly, the phonograph had “heard” hardly anything. As Kolkowski explained, this happened often in the past as well, which is why early recordists placed caged birds inside the horn in order to capture their sounds. Because the horn plunged the birds into darkness, however, they often refused to sing. This inspired the recordists to bring bird impersonators or flautists to the recording studios, just in case. We had a piccolo player, Anne Davids , as fallback option, and she beautifully played transcriptions of bird sounds as well as the flute score from Olivier Messiaen ’s “Le Merle noir” (The Blackbird ), originally composed for flute and piano. This time, the phonograph succeeded in recording and audibly replaying the music we had listened to. We recognized the tunes.

Although the demonstration had partially failed, the event was a success in terms of reenacting early twentieth-century bird recording. It conveyed to the audience what had been at stake at the time and articulated the sonic skills that were involved: heating the wax to the right temperature, turning the cylinder in such a way that it would run at the right speed (leading to the right recording pitch), adjusting the distance between sound source and phonograph , using the correct needles, and having the recorded subjects behave in preferred ways. Listening to birds through phonographic recording clearly entailed an intriguingly wide range of embodied forms of knowledge.

Showing this complexity of sonic skills in a performance for recordist and sounding subjects, the demonstration also underpinned Jonathan Sterne ’s claim that recording “is a form of exteriority: it does not preserve a preexisting sonic event as it happens so much as it creates and organizes events for the possibility of preservation and repetition” (Sterne 2003: 332; see also Bronfman 2016: 228; Brady 1999: 6–7). Sonic examinations, like other ways of staging phenomena to be recorded, redefined the researching experts in terms of the skills required, but they also redefined the objects studied. Things, animals, and humans were made to sound loud enough to be captured by the vibrating needles, membranes , and amplifying tools of their time. In some cases, subjects and researchers closely co-operated in creating the recordings. As Erika Brady has illustrated for ethnology , the humans under study at times only collaborated with researchers in ways that allowed them to remain true to their own cultural and epistemic conventions, thus impacting on what was recorded (Brady 1999: 111–117). The weight of the recording instruments and means of transportation affected which sounding objects could be reached. Different microphones resulted in different permeations between sounding objects and their environment, with huge effects on the questions posed, as we saw in the previous chapter. The sonic traces of those objects, and the resulting issues of interest were, as I will demonstrate below, even co-defined by the maximum length of the sound recordings or their visualizations.

In ornithology , mechanical sound recording did not remain the preferred medium for capturing and analyzing bird vocalization for long. Soon, ornithologists would rank the epistemological value of mechanically visualizing sound more highly than that of audio recording plus manual notation, at least for “diagnostic ” purposes as set out in the taxonomy of listening presented in Chapter 3. In automotive engineering, mechanical visualization replaced embodied listening as the portal to systematic diagnosis, earlier in the United States than in Germany. And in medicine, the situation differed for different specializations. How should this shifting balance between the senses be interpreted? Switching between sensory modalities, according to STS scholars Regula Burri et al. (2011), may be considered “an epistemic tactic with which the different senses are put into productive relations” (p. 4), but this does not yet account for the differences in when and how experts in particular fields switch from one sense to another.

Varying relations of trust between knowledge makers and knowledge takers will be one of the factors; the ecology of diagnostic instruments in which new technologies intervene is another. By following cases over time, I am able to show how, and under which conditions, practices of listening were critiqued or replaced by other techniques of knowing in science, medicine, and engineering. In terms of the modes of listening discussed in the previous chapter, the most explicit and heated discussions focused on diagnostic listening as opposed to monitory or exploratory listening . As I will explain in Chapter 5, this does not imply that the aims of monitory and exploratory listening diminished in significance in the sciences over time—quite the contrary. But attempts to mechanize the analysis of data through visualization centered on diagnosis.

Replacing embodied listening to sound by automatically recording sound, and subsequently by the mechanical visualization of sound, I will additionally claim, required processes of sensory calibration . 2 That is, new sensory practices had to be “anchored” in existing ones in order to acquire authority . Just as some music and dance researchers had tried to enhance the reliability of the manual notation of sound by calibrating it with the embodied, kinetic experience of music making and dancing, so the promoters of automatic registration and visualization of sound aimed to augment the authority of these practices by materially and metaphorically calibrating it with known sensory knowledge practices.

Such attempts at calibration were not always successful. The sonification community, for instance, hovered between presenting data visualization as a practice with which sonification had much in common and as one from which it should depart. But this rhetorical flexibility did, as yet, nothing to help the community find a convincing “killer application ”.

Beyond the Infallible Ear: Repeated Listening and Sensory Calibration

In Chapter 2, I discussed several scholarly fields that (with more or less initial hesitation) embraced the phonograph , most notably for its mimetic and mnemonic functions . In 1931, the Romanian folklorist Constantin Brăiloiu recapitulated these advantages when he expressed his trust in the phonograph ’s recording diaphragm as “infallible ear” (p. 394) in the essay “Outline of a Method of Musical Folklore ”:

The concern for objectivity impels us, first of all, to undertake the mechanical recording of melodies. Only the machine is objective beyond question and only its reproduction is unquestionable and complete. No matter how well we notate a performer’s melody by dictation, we will always miss something in our notation, whether it be the timbre of the voice of that particular coloration of the melody due to the vocal production of the peasant, [or] … the timbre of the instruments . Furthermore, the mechanical recording avoids fatiguing the informants, and facilitates an extensive collection. Finally, it provides us with a means of control which no exact science can do without. (Brăiloiu 1970/1931: 393)

By asserting the accuracy and reliability of mechanical sound recording, Brăiloiu voiced the same ideals as the ornithologists had done. Similarly, creating extensive collections and boosting productivity through phonographic recording was no less important in ornithology as it was in folklore research and ethnology (Brady 1999: 67). The only difference was that the naturalists did not have to worry about fatiguing their informants. Birds in the field could not care less.

Mechanical sound recording was not free of problems, however. I touched on some of these when discussing ornithology : the need for heavy equipment, motorized transportation, and electricity affected the recordability of the field , and the distance between bird and microphone in natural settings generated a dichotomy between atmospheric and close-up recordings. The folklorist Brăiloiu also mentioned that wax cylinders crumbled easily and deteriorated quickly; they did not “withstand more than twenty to twenty-five playings” and thus had to be transferred to discs if possible. He also thought it wise to have the infallible ear collaborate with “the infallible eye of the lens.” The engravings in wax or ebonite might “keep a flicker of life” and be “easier to comprehend” if auxiliary documentation such as photographs, films, and index cards with detailed descriptions of the singers and their ritual events were stored as well (Brăiloiu 1970/1931: 393–394, 397). For Brăiloiu, rural music was especially interesting as a living tradition, “the dual effort of integration and adaptation which tends, on the one hand, to pour the attributes of modern civilization into the mold of the tradition, and on the other hand, to impose upon this tradition the appearance of the contemporary world” (p. 392).

At least in Brăiloiu’s view, producing knowledge from sound recordings meant aligning and anchoring them in visual documentation beyond musical notation. Such forms of material sensory calibration were also crucial in ornithology . Joeri Bruyninckx has shown how the Cornell Library of Natural Sounds attempted to make volunteers’ recordings as valuable to research as professionals’ recordings. This aim acquired particular significance in the 1950s, when the magnetic tape recorder —less heavy , expensive, and cumbersome to use than the gramophone disc-cutter —became available to amateur sound hunters, while ornithologists ’ growing interest in population ecology and ethology made them keen to collect the greatest possible variety of bird recordings. The Library tried to ensure the accuracy and reliability of recordings by asking amateurs , for instance in its Bioacoustics Bulletin, to fill out standardized forms with information on the recorded bird species as well as the date, place, and other conditions of recording, and to use pitch pipes with a standardized tone of 440 Hz. Taping this tone before the bird ’s vocalization “enabled future users to calibrate their playback equipment and to detect deviations in recording speed” (Bruyninckx 2015: 358). Ornithologists thus tried to materially calibrate sound recordings both through existing sounding instruments, such as pitch pipes , and through the existing visual format of the standardized form. Analytic and synthetic listening for comparative diagnostic purposes was only considered possible through calibration and standardization of recording practices.

Encouraging amateurs to contribute to sensory calibration was only one aspect of the exchange relation that the Library developed with its volunteers. The institution shored up that relationship with various forms of capital. Social capital was exchanged when advice on making field records and expertise on the recorded birds was offered in return for the moral obligation to share information with the Library. Sponsoring field trips with travel funds and equipment in return for the recordings and the associated copyrights was an exchange of economic capital. And granting the amateurs authorship of the recordings and the distinguished status of research associates brought symbolic capital into play. Bruyninckx (2015) has explained in these Bourdieusian terms how the Library secured volunteers’ prolonged investment in its undertaking.

The sheer number of sound recordings collected, and the time it took to listen to them repeatedly, also prompted scientists to ask what exactly the recordings added to analysis beyond mere documentation. In this, they showed similarities with another group of professionals. As media scholar Tom Willaert has shown, several literary writers and intellectuals in late nineteenth- and early twentieth-century Flanders and the Netherlands felt the need to explicitly define their creative contribution now that the phonograph had proved capable of capturing verbatim language—what was left for writers to do? One such writer, the Dutch novelist Multatuli (that is, Eduard Douwes Dekker ), welcomed the phonograph as the perfect embodiment of his own poetics: his aspiration to natural-sounding language and an associatively unfolding argument. Others saw phonographic writing as a form of mechanical registration that might be allowable for entertainment literature, but must be kept out of high literature at all costs. Instead, writers should interpret and critically reflect upon what they observed; use exemplary language; and elaborate, contrast, and synthesize at proper places in their texts (Willaert 2016: 1–22, 33–41).

Scientists and scholars, too, were eager to underline what their expertise could add to mere registration. Rather than highlighting their subjectivity, as some literary writers had done, most cited their capacity to compare sounds systematically through repeated listening. Playing recordings at reduced speed was considered helpful in this process. In the 1960s and 1970s, several bird sound recordists tried to support the epistemic authority of this technique, in their case executed with gramophones , by comparing recorded sounds with microscopic images—just as the ethnomusicologist Benjamin Ives Gilman had done decades earlier when writing about “magnifying” the sound of his phonograph recordings. The technique allowed ornithological recordists to extend sonic fragments, so they claimed, just as microscopy enabled scientists to enlarge visual details (Bruyninckx 2018). Such a rhetoric intervention entailed a metaphorical rather than a material calibration with a known visual instrument and its sensory enhancement. Whether material or metaphorical, however, the processes of sensory calibration had to improve the reliability and authority of sound recording in the fields that used it.

At a Glance: Mechanical Visualization and Professional Audition

The world first learned about the sound spectrograph in 1947. That was the year when Ralf Potter, Director of Transmission Research at Bell Telephone Laboratories, and his former colleagues George Kopp and Harriet Green published their report Visible Speech. Although the title expressed their primary interest in visualizing speech, the spectrograph permitted the visualization of sound more generally. Technologies for the transduction of sounds into images as such were by no means new. The nineteenth-century “oscillograph ,” for instance, was already able to visualize the frequency and intensity of pure tones across time, with intensity represented as the amplitude of the sound’s horizontally developing waveform. 3 But the spectrograph enabled its users to make images of complex tones or a spectrum of frequencies—the default situation in speech. These images, or “sound spectrograms ,” plotted time horizontally and frequency vertically, and displayed the intensity of sound as shades of darkness. 4 The assumption was that the sound spectrograph mimicked human sound perception more accurately than earlier instruments had done because the ear also processes sound in terms of different frequency components (Potter et al. 1947: 8–13; Mills 2010: 38). Potter and his colleagues seemed to reserve the word “sonogram ” for the speech spectrogram, but other early users distinguished less clearly between the two terms. Today, “sonogram ” predominantly refers to ultrasound images in medical practices.

The Visible Speech authors believed the sound spectrograph would be useful in teaching the deaf and hearing -impaired to speak by giving them feedback on the quality of their speech, and would enable them to understand telephone calls by reading sonograms of telephone conversations in real time, though they admitted to having no conclusive proof of whether this was really possible (p. 6). They also referred to military usages of the technology. Behind the scenes, the military was working on uses of the spectrograph for unscrambling telephone messages and identifying speakers through “voice printing ” (Fehr 2000; Broeders 2002; Mills 2010: 52). At the end of Visible Speech, the authors speculated about an impressive series of possible future uses, including options for research in bioacoustics :

A little experience with the patterns of animal, bird, and insect sounds suggests that there may be fascinating possibilities of analysis, illustration and discussion that have not been available in the past. For example, permanent patterns of bird songs can be examined in great detail, and there is reason to believe that song differences within one group could be readily identified by visual comparisons. In fact, it would not be surprising if the song habits of individual birds could be recognized by close examination of the patterns. … The song becomes a signature!

If detailed analysis of song patterns is possible, there would seem to be a wide new field of study open to the ornithologist . Perhaps bird books and periodicals of the future will be filled with song pictures, and serious readers may become well enough acquainted with this sound language to read visible patterns of bird music in the way a musician reads a musical score. (Potter et al. 1947: 410–411)

From the late 1950s, ornithological journals would indeed be full of sound spectrograms . In the early years of that decade, the ethologists William Thorpe and Peter Marler had started experimenting with the technique, which was quickly taken up by the world of ornithology . In contrast to mechanical sound recording, the spectrograph could only represent a few—two to four—seconds of sound. Nevertheless, it was welcomed for its capacity to visualize pitches far beyond human hearing and to cope with the high speed and frequent modulations of bird vocalizations (Bruyninckx 2013: 119ff). In fact, it was these characteristics that fostered ornithologists ’ interest in studying bird calls—short in length, high in pitch , potentially rapid in repetition—instead of more extended bird vocalizations, another example of the knowledge effects of new instruments . The spectrograph also contributed to a focus on variation among other short elements in the repertoires of birds : did such variation signify new species, learned behavior, functional characteristics, adaptations to environmental change, or crucial forms of communication (Bruyninckx 2013: 160–161)? Looking back in 2004, Peter Marler —probably unintentionally—copied the rhetorical strategies of some of his predecessors in ornithology by comparing the sound spectrograph to the microscope (Marler 2004, cited by Bruyninckx 2013: 119). His comparison was yet another instance of metaphorical sensory calibration , as well as an expression of a further version of mechanical objectivity —leaving the ear out of a job not only in the field , but also in the office where the transcription phase took place.

Meanwhile, ethnomusicologists preferred the melograph to the spectrograph, because it produced two line graphs, one representing pitch across time and the other volume across time, creating an easily readable visualization of melodic, and especially vocal melodic lines. The melograph ’s inventor, the American musicologist Charles Seeger , had imagined it in the 1930s, but realized its potential only in the 1940s thanks to Potter’s technical drawings of the sound spectrograph , and had its first model built in the 1950s (Mundy 2018: 133; Prescatello 1992: 212). Despite the differences between the melograph and the spectrograph, both instruments produced visualizations of sound that facilitated the distribution of research data through publications on paper, as well as fruitful combinations of spectrograms with other forms of visual representation . Seeger advised his peers to add musical notation to the graphs, for instance, so that the notation would prescribe to readers how to sing a particular song while the graphs would describe how it had actually been performed (Prescatello 1992: 212–217; Mundy 2018: 215). And as Bruyninckx explains with the help of Bruno Latour ’s (1986) work on inscriptions , such images were mobile, immutable (or at least less vulnerable to erasure and alteration than sound recordings), flat, and thus easily “overseen, cut up, scaled, recombined or superimposed” (Bruyninckx 2013: 144). Whereas listening to sound recordings required a substantial time investment for each playing, sound spectrograms could be transformed into optically consistent images that could then be synoptically presented so as to compare the images at a glance and cascade them into other abstractions, such as numbers.

Bruyninckx adds, however, that the sound spectrograph did not achieve “instantaneous intelligibility ” (2013: 121). Some ornithologists began to abstract the sound spectrograms into calligraphic signs that they thought were more easily readable and printable and better represented the essence of the patterns observed. Masking the spectrograph’s visualizations of background noises with white paint was part of that procedure. Others expressed their dislike of such interventions—or indeed any interventions. Still others added verbal descriptions and syllabic notations to the sound spectrograms , hoping to give each other “an impression of how the sounds might appear ‘to the human ear.’” Such subjective accounts, Bruyninckx claims, did not only supply “information that could not be conveyed otherwise”; they also marked “a perceptive minimum to orient the observations of other ornithologists” (2013: 135). Offering such manually notated aural information thus assisted the analysis of bird sound by peers, and again anchored a new visual technology in earlier techniques.

Issues of legibility and analysis were also raised by ornithologist Donald J. Borror in 1956. He noted that it was very hard to define the beginning and the end of Carolina wren song phrases just by looking at sound spectrogram . Because delimiting phrases was crucial for his work, he had to combine the imagery with listening to sound recordings at reduced speed. In the early 1970s, Robert Lemon , a biologist working at McGill , relied on the auditory sense in the same way. In his study of how cardinals responded to the prerecorded songs of other cardinals, he mentioned that although all new recordings of their vocal behavior had been analyzed with help of a sound spectrograph , “much information …, especially relating to the sequences of different song types, was gathered by listening to the birds sing and then recording the data in a notebook” (Lemon and Chatfield 1971: 1, cited by Bruyninckx 2013: 135). In this way, both Borror and Lemon underpinned the understanding of sound spectrograms with analytic listening.

Some ornithologists even started combining sound spectrograms with musical notation. A few did so because they refused to abandon the search for musical patterns in bird vocalizations, a search that was by then highly contested. Others argued that the linear frequency scale of sound spectrograms did not express the logarithmic sensation of pitch to which both humans and birds responded, or else considered musical scores more accessible and comprehensible than spectrographic images.

For these ornithologists , musical skills remained an integral part of the ornithologist ’s expertise , or, following Bruyninckx’s (2013: 41–42) reference to Thomas Porcello , of the ornithologist ’s “professional audition .” Thomas Porcello coined this term in 2004, based on Charles Goodwin ’s 1994 notion of “professional vision ,” to capture the discursive and embodied competences of experienced sound studio engineers —expressed in their shared technical, musical, and linguistic repertoire for bringing together audio technologies , techniques, and sonic ideals—and to emphasize novices’ lack of such authoritative and efficient expertise .

At another level, Bruyninckx argues (2013: 134), abstracting spectrograms, adding manual notations to automatically generated spectrographic images, and carrying out listening exercises meant the introduction to ornithology of what Lorraine Daston and Peter Galison (1992, 2007: 309–361) have called “trained judgment .” Trained judgment represents the gradual enrichment of the ideal of mechanical objectivity in the sciences by confidence in the sophistication of experts ’ interpretive skills, drawing on an intuitive and holistic understanding of data patterns developed through years of experience. But it also entails judgment of when exactly such interpretive skills should inform analysis and when they should not.

Reintroducing musical notation, for instance, was considered unacceptable by the large majority of ornithologists. True, some of them conceded, the composer Messiaen had successfully simulated bird song with such notations—but using them in biology would mean wrongly assuming that birds were musical creatures (Bruyninckx 2013: 141).

Meter Reading as a Technology of Trust in Expert–Customer Relations

Although sound spectrograms made their presence felt in bird books and periodicals in the 1950s and after, by then ornithology had already experienced a golden age of sound recording—including in commercial terms, with large sales for the records issued by the Cornell Library of Natural Sounds . 5 Other branches of bioacoustics , as well as ethnomusicology , ethnology , and linguistic fields such as dialectology and phonology , also embraced sound recording as a token of advanced scholarship for a substantial period of time.

In the “stethoscopic ” fields of medicine and automotive engineering, mechanical sound recording acquired different roles. In medicine, gramophone records enabled physicians to document and circulate rare cases, and were also used in teaching contexts. As early as 1930, a renowned American textbook on physical diagnosis referred to a series of Columbia records that students could listen to repeatedly. But the medical field did not unanimously embrace listening to the body through loudspeakers . In the 1940s, several authors asserted that the technology had a detrimental effect on the isolating and filtering experience of listening with the stethoscope, as background noise and loudspeaker buzzing interfered, destroying—in Van Drie ’s rendering of these comments—“the impression of the closed acoustic pathway ” (2013: 178) that had been the stethoscope’s great advantage. One solution was to have a professor in a lecture hall carry out a auscultation live with an electronic stethoscope , then broadcast it to students who had individual stethoscopes plugged into the transmission system, thus reinstating the closed pathway experience. Another was to publish “teaching tapes ” (or audiocassette recordings in the 1970s) specially designed to be listened to by holding the stethoscope’s bell a few inches from the speaker of the tape recorder . Employed in this way, the recorder acted as an “electronic chest ” (p. 180).

Even so, mechanical sound recording was less widely used for diagnostic listening in medicine than in fields such as ornithology , ethnomusicology , and phonology . It is no surprise that medicine differs from areas in which sound, as music or language, is the key object of study. But for ornithology , the dissimilarity with medicine is less self-explanatory. To understand it, we need to take the field -specific ecology of diagnostic instruments into account. Most medical specializations could draw on both the stethoscope and visual diagnostic techniques such as radiology before mechanical sound recording entered the scene. This made mechanical sound recording less essential for diagnostic listening , although it was not entirely absent. In the mid-1960s, for instance, slow-speed magnetic tape recording was used for the objective assessment of “cough suppressants under clinical conditions” (Woolf and Rosenberg 1964); three decades later, visualizing high-speed magnetic tape recordings of cough sounds was ascribed “considerable value in identifying mechanisms of airway pathology present in respiratory diseases” (Korpáš et al. 1996: 261). Indeed, biomedical acoustics has become a highly developed field . For the most part, however, it focuses on visualizing ultrasound rather than sound in the human auditory range . So while the stethoscope lived on in the ward, and mechanical sound recordings played a modest role in medical educational settings, it was imaging that dominated diagnostic work in behind-the-scenes hospital laboratories.

Stefan Krebs has shown how the transition from auditory to visual diagnostics unfolded in car mechanics in Germany and the United States. As discussed in previous chapters, German mechanics modeled their diagnostic listening skills on the medical world, using automotive stethoscopes as supplements to screwdrivers and listening rods to focus their listening. When confronted with a repair crisis in the 1930s, they did not follow their US colleagues in introducing meters and gauges to restore a relation of trust with customers, but embedded car mechanics in a system of certified guilds . Whereas in the United States, the damage to customers’ belief in the capacities of mechanics had been mitigated with legible, and thus visual, instruments as technologies of trust , in Germany the long-standing tradition of crafts guilds underpinned mechanics’ automotive authority . Diagnostic listening could become an insignia of the German mechanics’ profession, and came to stand “metonymically for all embodied sensory skills car mechanics developed through training and expertise ” (Krebs 2014: 355; see also Krebs and Van Drie 2014: 100). After World War II , the American military seized the opportunity to replace the restrictive German guilds by freedom of trade in the zone they controlled. Indicating the deep roots of the guilds in German society, however, the West German government reestablished the system in 1953.

Because of this history, new diagnostic instruments entered the German garage some twenty years later than the American one. Germany´s leading manufacturer of car systems, Robert Bosch, had introduced a test instrument for spark plugs as early as World War I , but only used this and similar devices for its own services. When it did start selling diagnostic tools to German repair shops in the 1950s, the dominant argument was not that the devices would increase demand for repairs, as had been the US selling strategy, but that they would save time by helping mechanics to find the cause of troubles without having to disassemble the car. This efficiency argument was important because the German repair world was struggling with a shortage of car mechanics , who could earn much more in the rapidly expanding automotive industry than in repair shops. Other arguments focused on the automotive technology itself—electrical systems being increasingly sophisticated and more highly powered machines permitting smaller tolerances—and on a new form of after-sales service that was geared towards preventing car problems rather than solving them. One article in an automotive journal additionally played the objectivity card. It deployed the medical metaphor again, but this time to advocate visual instead of audible tools : American diagnostic devices were superior to the mechanic ’s senses , and the tools were “like medical instruments in an operating room, covered in white enamel and chrome, to be wheeled silently towards the patient: the ‘sick’ automobile ” (“Sie fragen,” 1957: 43–44, cited by Krebs 2014: 367).

From the late 1960s onwards, car manufacturers offered their garages diagnostic test stands and increasingly required them to be used. In parallel, the trade press criticized traditional methods: “You can no longer master modern automobiles with your expert senses ; only with modern diagnostic instruments is it possible to do the necessary tests and adjustments ” (“Prüfen,” 1966: 20, cited in Krebs 2014: 370). Bosch also pitted conservative experts against progressive ones, who preferred “measuring instead of guessing, checking instead of trying, and testing instead of sensing.” 6 Such rhetorical moves towards objectivity, Krebs argues, were entangled with a wider visual culture. Bosch presented the oscilloscope (an oscillograph with a screen), for example, as the mechanic ’s television (Krebs 2014: 372).

German mechanics did not give in easily, however. They resisted by claiming that American-style diagnostics would produce mechanics who were able merely to change parts, not to truly repair a car. To them, sensory skills were more than filing, drilling, lathing, forging, and listening: they also encompassed tidiness, punctuality, meticulousness, and care. Such skills were considered crucial in mechanics’ relationship with customers. An expert mechanic would, for instance, be able to step beyond the standards imposed by manufacturers and “increase the specified valve clearance by another 0.10 mm to avoid valve ticking that often annoyed drivers ” (Krebs 2014: 376). Moreover , they had complaints, at times justified, about the accuracy of the equipment, or simply wanted to prevent customers from noticing their inexperience in handling the devices. Only in the late 1970s did the situation begin to change, due to the rise of the electronically rather than mechanically functioning car, which strengthened the position of the manufacturers and repair shop employers at the expense of the mechanics’ jurisdiction over embodied skills.

Understanding such context-specific appropriation of visual diagnostic techniques can contribute to the present-day STS debate on tacit knowledge. Sociologist of science Harry Collins distinguishes between three forms of tacit knowledge : relational, somatic, and collective. Relational tacit knowledge refers to knowledge that is not explicit, formalized, or captured in rules, though only for the time being—once social relations change, such tacit knowledge may become explicit after all. Secrets are a case in point. Somatic tacit knowledge is embodied knowledge , hard to explain because of its incarnated character, but possibly explicable in the future; an example is riding a bike. Collective tacit knowledge , the most robust form of tacit knowledge, is knowledge that entirely resists explication because it can only be acquired by spending time with those who already have it; cycling in local traffic, for instance (Collins 2013). Rather than classifying tacit knowledge substantively as Collins does, Michael Lynch finds it more interesting to “examine what is done with the notion of ‘tacit knowledge’” as a polemical and professional resource (Lynch 2013: 58). Indeed, this and the previous chapters have shown how German car mechanics succeeded for quite some time in defining their auditory and other sensory expertise as knowledge that could only be learned by spending time with a senior member of their field , so as collective tacit knowledge . The auto-stethoscope was a token of their professional autonomy until labor shortages, diminishing tolerances, ideals of preventive care, and electronically steered cars arrived and there was less and less to be heard. 7

There Is More Than Meets the Eye: Struggling with Trained Ears in Sonification

The community of researchers who have been promoting data sonification since the early 1990s have a deeply ambiguous relationship with the phenomenon of data visualization . As Alexandra Supper has shown, sonification researchers have passionately lobbied for the ear by positioning sonification as a much-needed alternative to visualization . Auditory displays of data, so they commonly claim, allow for an easier recognition of patterns than visual data presentations. But they have also stressed sonification ’s similarity to visualization : just as graphs and diagrams are conventional representations of data that have gained authority over time, sonification deserves, as a convention-in-the-making, to gradually acquire acclaim in the academic world. For sonification enthusiasts, visualization is thus both a phenomenon to depart from and to set as example (Supper 2012a: 264; 2016).

An often used argument for sonification is that the huge amount and widespread availability of digitalized data in science and society today call for new ways of processing. So far, “data exploration tools ” have been predominantly “visual in nature, including graphing and plotting software, modeling programs, and 2[D] and 3D visualization software,” notes psychologist and computer scientist Bruce Walker , but these tools “fail to exploit the excellent pattern recognition capabilities of the human auditory system, and they also continue to exclude students and researchers with visual disabilities ” (Walker 2000: 16f, cited in Supper 2012b: 17). In the words of system biologist Peter Larsen, who has translated data from microbial ecology into a jazz composition, “there is only so much” that a person or even a computer “can do to see patterns in these outrageously huge data sets” (Larsen cited by Brannen 2013: 1, in turn cited by Supper 2015a: 441).

Makers of sonifications consider auditory data display to be especially relevant for those sciences that already work with data based on vibration or oscillation , such as seismology , volcanology , or astronomy because these data are reasonably easy to transfer to the human auditory range . Sonification is also presented as very interesting for sciences that produce data with a temporal dimension, for example electroencephalograms (EEGs) of epileptic seizures. Sonifications of EEGs, their proponents argue, have rhythmic and spatial dimensions that may make the dynamics of seizures more intelligible (Supper 2012b: 14–15).

Given the presumed analytical advantages of listening to data, members of the sonification community find it unfair that the ear has been taken less seriously than the eye in the production of knowledge. This makes them keen to refer to auditory activities they see as precursors to their work. In doing so, they make no explicit distinction between listening to sounds that occur “naturally or as an unintended byproduct of another activity” and to sounds that have been “deliberately made for the purposes of revealing information” (Supper 2015a: 445). Evidently, listening to the second type of sound is closer to sonification proper, but it is a rhetorically effective move to bracket both types of sound together. The use of stethoscopes in medicine and automotive engineering is a long-standing example of the first kind of listening; an instance of the second kind is the Geiger counter. Both cases are often referred to in sonification literature. Another example enthusiastically embraced by the sonification community is psychoacoustician Sheridan Speeth ’s 1960s digital transposition of seismic signals into sound “in an effort to find a reliable method to distinguish earthquakes from underground nuclear explosions” (Supper 2015a: 446, see also Volmar 2013). In all these cases, historical examples of listening are cited to undermine the presumed dominance of visual modes of analyzing and presenting data.

Proponents of sonification do not always write antagonistically about visualization , however. In their arguments, visualization is sometimes also an authoritative practice prefiguring and acting as a model for sonification . Thus, the claim that “subjective decisions are widely accepted in data visualization ” may be used to argue that such interventions are also acceptable in sonification (Supper 2012c: 32). Whereas some members of the sonification community, notably psychologists, ask for quantitative user tests to substantiate “what the average listener actually hears in a sonification ,” more theoretically inclined sonification researchers deplore such demands (Supper 2012c: 31). Visualization , they point out, is commonly used without scientists requesting user tests showing that it actually works in conveying information, so why should user tests be necessary for sonification ? If visualization has organically developed into an accepted set of conventions, why not grant the same evolution to sonification ? As one of these promoters has it, the “first visualizations of molecules were not evaluated, they were just made. And they were extremely functional ” (Interview Florian Grond, cited in Supper 2012a: 263).

Their ideal is to rely on what Supper calls the “trained ears ” of sonification designers and the experts , or domain scientists, whose data are sonified. Supper’s formulation plays with Daston and Galison’s notion of “trained judgment .” In the sonification community, it stands for the position that mapping data onto particular sound parameters always implies choice and subjectivity, but that this is no problem as long as sonification designers and domain scientists are willing to listen to different mappings of the same data sets, which represent different “(sonic) views,” to quote Thomas Hermann , a prominent member of the community, and his co-authors (Hermann et al. 2007: 467 cited in Supper 2012c: 32).

Views on visualization among sonification researchers experts have also changed over time. Whereas several of the sonification movement’s pioneers expressed the hope that sonification would replace visualization in the long term, most sonification researchers today find it more realistic to present sonification as an important add-on to the range of data representation tools already available. Better still, they would like to collaborate not only with experts who are already used to listening, such as physicians, but also with domain scientists who commonly work with digital visualization techniques. By creating sonification plug-ins for a widely used data visualization software package, for instance, they enable domain experts to rely on their existing expertise and skills rather than being forced to familiarize themselves with the software used by sonification designers (Supper 2012a: 256; 2015a: 453).

In fact, when presenting their own work at conferences, members of the sonification community do not eschew diagrams, photos, or screen shots. And although the organizers of each International Conference on Auditory Display ask for sound examples, a recurrent complaint is that these are little used even in this heartland of sonification , and then mainly by the most experienced sonification proponents. This is partially due to the high likelihood of errors when using sound, fear of criticism regarding sound quality, or bad listening conditions in the venues. Instead, “data karaoke ”—a phenomenon mentioned in Chapter 2—is often used to embody, highlight, illustrate, and authorize data and/or sonifications or to integrate them with the sonic form of the spoken presentation itself (Supper 2015b).

When claiming a place in the academic sun for sonification , Supper (2012a) has argued, the sonification community engages in “boundary work ” as members shift rhetorically between connecting their work with and distancing it from visualization , depending on the contexts in which sonification is being propounded. Supper draws on STS findings on the ways that academic fields construct their cultural authority by demarcating themselves from non-science such as politics and religion (Gieryn 1995), crossing the borders of other disciplines (Klein 1996; Burri 2008), defining the conditions under which existing boundaries can be crossed (Halffman 2003), or showing how an emerging field differs from and resembles established disciplines (Amsterdamska 2005). As we will see in the next chapter, sonification promoters have used similar strategies when positioning sonification in relation to art and music.

Supper also stresses, however, that the “current situation in sonification echoes, rather than solves” the problem of finding information in large datasets (Supper 2015a: 458). As she has amply documented, most work in sonification focuses on tools and designs for sonification rather than the analysis of data. Critics within the community complain that most of their colleagues seem to think these information patterns simply “jump out” from the data once the sonification has been made. Again, the sonification community’s preferred choice of historical examples is canny. The Geiger counter displays levels of a well-known scientific phenomenon, while “the audification of earthquakes and nuclear detonations rests upon the principle of pattern recognitions in instances where the patterns themselves are well understood” (Supper 2015a: 456). For sonifications of the big data sets that sonification researchers have in mind, in contrast, the patterns remain to be found and understood, and do not make themselves apparent automatically.

Moreover, sustained collaboration and shared listening would be necessary for the datasets to be properly understood by sonification specialists and the sounds to be properly understood by the domain scientists. This is still rare (Supper 2015a: 457). Partially as a result, the sonification community has not yet found its “killer application ” for diagnostic listening. In informatics, the term killer application “refers to an application program so useful that users are willing to buy the hardware it runs on, just to have that program” (Juolo 2008: 76, cited in Supper 2012a: 255). So far, and despite the success of the EEG sonifications, no sonification application has attracted enough interest for domain scientists to buy into sonification on that scale—a situation deeply deplored by sonification researchers.

Conclusions

This chapter has explained the conditions under which embodied listening, notably for purposes of “diagnostic listening,” to mechanically registered auditory phenomena was replaced by the mechanical analysis and visual display of data as the preferred sensory mode for producing knowledge. In the 1920s and 1930s, enthusiasm for mechanical sound recording as the epitome of mechanical objectivity reached a peak in many scientific fields. But as a solution to problems of sensory subjectivity, this enthusiasm for—or epistemological value projected onto—sound recording proved rather short-lived. The spectrograph promised to sidestep the cumbersome and controversial manual notation of sound , and enabled easier integration with texts and other visualizations on paper. In many medical fields, the tape recorder never really took off, as other auditory and visual instruments of diagnostics had already nested in medical practice.

In both the heydays of mechanical sound recording and the spectrograph, new instruments acquired scientific authority by being calibrated according to ways of listening or visual modalities that had been used in earlier periods. The sensory calibration of sound in vision, or vice versa, could be material in character, requiring an alignment with tangible sources captured in the same or other sensory modes , or metaphorical, suggesting similarities with modalities employed in the past: sound spectrograms conceptualized as a microscopy of sound, for instance. For some ornithologists , spectrography could not convey scientifically relevant findings without manual inventions and without anchoring the visualizations in comparative analytic listening . It was feared that particular phenomena, such as phrases in bird song , would be lost if the visualization of data through sound spectrograms was not enriched by professional audition. After all, a scientist was more than just a registrant. Trained judgment was those ornithologists’ ideal, complementing the notion of mechanical objectivity . At stake here was professional audition ’s authority to have a say in the establishment of knowledge. That was also the goal some researchers in the sonification community had in mind. They underlined the similarities between trained judgment in sonification and visualization . As yet, however, they have failed to gain significant ground for collaborative work on sonification in the world of science at large, perhaps because they also flag the differences between sonification and visualization .

In medicine and car mechanics, the move was not so much from mechanical sound recording to mechanical visual recording as from stethoscopic listening to meter reading . In US automotive mechanics , this happened much earlier than in Germany, a discrepancy that can only be understood by considering the changing relations of trust between car mechanics and motorists. While the state-backed crafts system kept German car mechanics the authorized and reliable owners of diagnostic listening, the social setting in the United States in the long run made meter reading the only way to assure customer trust in car mechanics. In Germany, the same shift was completed only in the late 1970s, with rationalization and the electrification of the car as arguments. Even then, the use of sound and listening for knowledge acquisition continued to crop up, or “to pop up” in the sciences. What should we make of this seemingly stubborn reappearances?

Notes

  1. 1.

    See “Bird organ” under “Bird instruments” on Oxford Music Online. The first serinettes were built in seventeenth-century France (serin is a French word for canary), but the instruments were most popular in the eighteenth and nineteenth century, also outside France. The Oxford Music Online entry claims that serinettes were used to “encourage” canaries to sing. In 1852, however, the French bird breeder Jules Jannin explained that serinettes could also be employed to teach caged canaries to sing particular songs (Jannin 1852: 27).

  2. 2.

    I would like to thank Cyrus Mody for suggesting this line of thought.

  3. 3.

    An oscillograph is “a device that generate[s] visual displays of electrical signals” (Thompson 2002: 96). It can be used to indicate, in waveform, any quantity that can be converted into electric energy. One such quantity is acoustic energy.

  4. 4.

    At that time, a sound spectrograph deciphered the sound signals of a magnetic sound recording by measuring the sound energy of particular frequency ranges in that signal with a frequency band filter. A stylus recorded the sound energy in each of the frequency bands on a “revolving roll of electrically sensitive paper” (Bruyninckx 2013: 123). For spectrographic images, see http://exhibition.sonicskills.org/exhibition/booth4/graphical-notation-the-spectrograph/ and http://exhibition.sonicskills.org/exhibition/booth4/notating-bird-song-and-sound/ (both last accessed August 14, 2017). On the second link, scroll down to “Transcriptions of chaffinch song,” last page.

  5. 5.

    In 1958, the Laboratory of Ornithology at Cornell University , the institute behind the Cornell Library of Natural Sounds , earned 10,000 dollars from royalties. The institute published its own record series and sound books: natural history books with aural illustrations on gramophone . It also sold its recordings as sound effects to the entertainment industry, including Disney and Warner Bros, and to businesses, which played the sounds of natural predators in pest control, for example (Bruyninckx 2015: 353).

  6. 6.

    Robert Bosch Company Archives, “Die ganze Werkstatt-Ausrüstung Bosch,” 1969, File Number EF 001/009.

  7. 7.

    Michael Lynch (2013: 68) refers to a 1985 work by historian of medicine Christopher Lawrence showing that some late nineteenth-century Victorian gentlemen-doctors even considered the stethoscope to threaten the standing of their “incommunicable knowledge,” because it might open the medical profession to specialists beyond their own elite ranks.