Skip to main content
SpringerLink
Log in

Sound, Ecological Affordances and Embodied Mappings in Auditory Display

  • Chapter
  • First Online:
Book cover New Directions in Third Wave Human-Computer Interaction: Volume 2 - Methodologies

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

The third wave of HCI has seen the widespread adoption of design principles borrowed from and informed by breakthroughs in the field of embodied cognitive science. These developments have taken place primarily in the contexts of visual media and interaction, but they are also of importance to the design of auditory displays and interactive systems in which sounds plays a dominant role, where they open up new affordances by which information might be communicated to a listener. This chapter examines the relationship between auditory display, sonic interaction design and embodied cognition and explores frameworks from embodied cognition that might inform the design of more informative auditory displays in a variety of contexts. It will do so by addressing these issues from an interdisciplinary perspective, bringing together insights from cognitive science and philosophy, general HCI and computer science, along with music theory and practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Antle AN, Corness G, Bevans A (2011) Springboard: designing embodied schema based embodied interaction for an abstract domain. In: England D (ed) Human-computer interaction series: whole body interaction, Springer, Heidelberg, pp 7–18

    Google Scholar 

  • Armstrong DF, Stokoe WC, Wilcox SE (1995) Gesture and the nature of language. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Arnheim R (1954) Art and visual perception: a psychology of the creative eye. University of California Press, Los Angeles

    Google Scholar 

  • Arnheim R (1984) Perceptual dynamics in musical expression. Music Q 70(3):295–309

    Article  Google Scholar 

  • Bannon L (1991) From human factors to human actors: the role of psychology and human-computer interaction studies in system design. Des Work Coop Des Comp Systems 25:44

    Google Scholar 

  • Bannon LJ, Bødker S (1989) Beyond the interface: encountering artifacts in use. DAIMI Rep Ser 18:288

    Google Scholar 

  • Barass S, Jeon M, Fernström M (2018) Editorial introduction in ergonomics in design special issue: sonic information design (forthcoming)

    Google Scholar 

  • Barrass S (1998) Auditory information design. Dissertation, Australian National University, Canberra

    Google Scholar 

  • Barrass S (2012) The aesthetic turn in sonification towards a social and cultural medium. AI & Soc 27(2):177–181

    Article  Google Scholar 

  • Barrass S, Vickers P (2011) Sonification design and aesthetics. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 145–171

    Google Scholar 

  • Barrett N (2015) Creating tangible spatial-musical images from physical performance gestures. In: Proceedings of the 2015 Conference on New Interfaces for Musical Expression (NIME 15), Louisiana State University, Baton Rouge, 31 May–3 June 2015

    Google Scholar 

  • Barsalou LW (2010) Grounded cognition: past, present, and future. Top Cogn Sci 2(4):716–724

    Article  Google Scholar 

  • Bødker S (2015) Third-wave HCI, 10 years later---participation and sharing. Interactions 22(5):24–31

    Article  Google Scholar 

  • Bødker S, Klokmose CN (2016) Dynamics, multiplicity and conceptual blends in HCI. In: Proceedings of the 2016 CHI conference on human factors in computing systems, San Jose, CA, 7–12 May 2016

    Google Scholar 

  • Bovermann T, Hermann T, Ritter H (2006) A tangible environment for ambient data representation. In: Proceedings of the first international workshop on haptic and audio interaction design, Glasgow, 31 August - 1 September 2006

    Google Scholar 

  • Brandt PA (2004) Spaces, domains, and meaning: essays in cognitive semiotics. In: Bax M, Van Heusden B, Wildgen W, Brandt PA (eds) European semiotics: language, cognition, and culture. 4, Peter Lang Publishing, Bern

    Google Scholar 

  • Brazil E, Fernström M (2006) Investigating concurrent auditory icon recognition. In: Proceedings of the 12th International Conference on Auditory Display (ICAD 2006), London, 20–23 June 2006

    Google Scholar 

  • Bregman AS (1990) Auditory scene analysis. MIT Press, Cambridge, MA

    Book  Google Scholar 

  • Bregman AS (1994) Auditory scene analysis: the perceptual organization of sound. In: McAdams S, Bigand E (eds) Thinking in sound: the cognitive psychology of human audition, vol 95. Oxford University Press, New York, p 1177

    Google Scholar 

  • Brooks R (2003) Robot: the future of flesh and machines. Penguin Books, London

    Google Scholar 

  • Brower C (2000) A cognitive theory of musical meaning. J Music Theory 44(2):323–379

    Article  Google Scholar 

  • Brown B, Aaron M (2001) The politics of nature. In: Smith J (ed) The rise of modern genomics, 3rd edn. Wiley, New York, pp 234–295

    Google Scholar 

  • Buxton WAS (1987) The haptic channel. In: Baecker RM, Buxton WAS (eds) Readings in human computer interaction: a multidisciplinary approach. Morgan Kaufmann, San Mateo, pp 357–365

    Google Scholar 

  • Card S, Moran T, Newell A (1986) The model human processor- an engineering model of human performance. Handb Percept Hum Perform 2:45–41

    Google Scholar 

  • Cook P (2001) Principles for designing computer music controllers. In: Proceedings of the 2001 conference on new interfaces for musical expression (NIME 2001), Seattle, WA, 1–2 April 2001

    Google Scholar 

  • Cox A (2001) The mimetic hypothesis and embodied musical meaning. Music Sci 5(2):195–212

    Article  Google Scholar 

  • Deacon T (2006) The aesthetic faculty. In: Turner M (ed) The artful mind: cognitive science and the riddle of human creativity. Oxford University Press, Oxford

    Google Scholar 

  • Dewey J (1934) Art as experience. Minton, Balch and Co., New York

    Google Scholar 

  • DeWitt A, Bresin R (2007) Sound design for affective interaction. In: International Conference on Affective Computing and Intelligent Interaction. Springer, Heidelberg, pp 523–533

    Chapter  Google Scholar 

  • Diniz N (2012) A multimodal framework for interactive sonification and sound-based communication. Dissertation, Ghent University, Belgium

    Google Scholar 

  • Diniz N, Deweppe A, Demey M Leman M (2010) A Framework for music-based interactive sonification. In: the Proceedings of the 16th International Conference on Auditory Display (ICAD 2010), Washington DC, 9–15 June 2010

    Google Scholar 

  • Diniz N, Coussement P, Deweppe A, Demey M, Leman M (2012) An embodied music cognition approach to multilevel interactive sonification. J Multimodal User Interfaces 5(3–4):211–219

    Article  Google Scholar 

  • Dourish P (2004) Where the action is: the foundations of embodied interaction. MIT Press, Cambridge, MA

    Google Scholar 

  • Dreyfus HL (1965) Alchemy and artificial intelligence. The Rand Corporation, Research Report, Santa Monica

    Google Scholar 

  • Droumeva M, de Castell S, Wakkary R (2007) Investigating sound intensity gradients as feedback for embodied learning. In: The Proceedings of the 13th International Conference for Auditory Display (ICAD 2013), Montreal, 26–29 June, 2007

    Google Scholar 

  • Droumeva M, Wakkary R (2008) Understanding aural fluency in auditory display design for ambient intelligent environments. In Proceedings of the 14th International Conference on Auditory Displays, Paris, France, June 24–27, 2008

    Google Scholar 

  • Faste T, Faste H (2012) Demystifying “design research”: Design is not research, research is design. In: The Proceedings of the industrial designers society of america education symposium, Boston, 15 August 2012

    Google Scholar 

  • Fauconnier G, Turner M (2002) The way we think: conceptual blending and the mind’s hidden complexities. Basic Books, New York

    Google Scholar 

  • Fencott R, Bryan-Kinns N (2010, June) Hey Man, You’re Invading my Personal Space! Privacy and Awareness in Collaborative Music. In NIME (pp. 198–203)

    Google Scholar 

  • Flowers JH (2005) Thirteen years of reflection on auditory graphing: promises, pitfalls, and potential new directions. In: The proceedings of the 11th International Conference on Auditory Display (ICAD 2005), Limerick, 6–9 July 2005

    Google Scholar 

  • Fodor J (1975) The language of thought. Thomas Y. Crowell, New York

    Google Scholar 

  • Gardner H (1985) The mind’s new science. Basic Books, New York

    Google Scholar 

  • Gaver WW (1989) The SonicFinder: an interface that uses auditory icons. Hum Comput Interaction 4(1):67–94

    Article  Google Scholar 

  • Gibson JJ (1977) The theory of affordances. In: Shaw R, Bransford J (eds) Perceiving, acting, and knowing: toward an ecological psychology. Lawrence Erlbaum, Hillsdale, pp 67–82

    Google Scholar 

  • Gibson JJ (1978) The ecological approach to the visual perception of pictures. Leonardo 11(3):227–235

    Article  Google Scholar 

  • Glenberg AM, Robertson DA (2000) Symbol grounding and meaning: a comparison of high-dimensional and embodied theories of meaning. J Mem Lang 43(3):379–401

    Article  Google Scholar 

  • Godøy RI (1997) Knowledge in music theory by shapes of musical objects and sound-producing actions. In: Leman M (ed) Music, gestalt, and computing. Springer, Berlin/Heidelberg, pp 89–102

    Chapter  Google Scholar 

  • Godøy RI (2003) Motor-mimetic music cognition. Leonardo 36(4):317–319

    Article  Google Scholar 

  • Godøy RI (2006) Gestural-sonorous objects: embodied extensions of Schaeffer’s conceptual apparatus. Org Sound 11(02):149–157

    Article  Google Scholar 

  • Gossmann J (2010) From metaphor to medium: sonification as extension of our body In: Proceedings of the16th International Conference on Auditory Display (ICAD2010), Washington, DC, June 9–15, 2010

    Google Scholar 

  • Graham R, Bridges B (2014) Gesture and embodied metaphor in spatial music performance Systems Design. In: Proceedings of the 2014 conference on New Interfaces for Musical Expression (NIME 14), Goldsmiths University of London, London 30 June - 3 July 2014

    Google Scholar 

  • Graham R, Bridges B (2015) Managing musical complexity with embodied metaphors. In: Proceedings of the 2015 conference on New Interfaces for Musical Expression (NIME 15), Louisiana State University, Baton Rouge, 31 May–3 June 2015

    Google Scholar 

  • Graham R, Manzione C, Bridges B, Brent W (2017) Exploring Pitch and Timbre through 3D spaces: embodied models in virtual reality as a basis for performance systems design. In: Proceedings of the 2017 conference on New Interfaces for Musical Expression (NIME 17), Copenhagen, 15 May–19 May 2017

    Google Scholar 

  • Grey JM (1977) Multidimensional perceptual scaling of musical timbres. J Acoust Soc Am 61(5):1270–1277

    Article  Google Scholar 

  • Grond F (2013) Listening–Mode–Centered Sonification Design for Data Exploration. Dissertation, Technische Fakultät der Universität Bielefeld, Bielefeld

    Google Scholar 

  • Grond F, Berger J (2011) Parameter mapping sonification. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 363–397

    Google Scholar 

  • Gurevich M, Treviño J (2007) Expression and its discontents: toward an ecology of musical creation. In Proceedings of the 7th International Conference on New Interfaces for Musical Expression (NIME 2007), New York University, New York, 6–10 June 2007

    Google Scholar 

  • Halpern AR, Zatorre RJ, Bouffard M, Johnson JA (2004) Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 42(9):1281–1292

    Article  Google Scholar 

  • Hampe B (2005) Image schemas in cognitive linguistics: introduction. In: Hampe B, Grady JE (eds) From perception to meaning: image schemas in cognitive linguistics. Walter de Gruyter, Berlin

    Chapter  Google Scholar 

  • Harnad S (1990) The symbol grounding problem. Physica D: Nonlinear Phenom 42(1):335–346

    Article  Google Scholar 

  • Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. In the Proceedings of Alt. Chi. Session at the SIGCHI conference on human factors in computing systems, San Jose, 28 April – 3 May 2007

    Google Scholar 

  • Heidegger M (1927) The basic problems of phenomenology. Indiana University Press, Bloomington

    Google Scholar 

  • Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: The proceedings of the 14th International Conference on Auditory Display (ICAD 2008), IRCAM Paris, 24–27 June 2008

    Google Scholar 

  • Hubbard TL (2010) Auditory imagery: empirical findings. Psychol Bull 136(2):302

    Article  Google Scholar 

  • Hunt A, Wanderley MM, Paradis M (2002) The importance of parameter mapping in electronic instrument design. J New Music Res 32(4):429–440

    Article  Google Scholar 

  • Hurtienne J (2009) Image schemas and design for intuitive use: exploring new guidance for user interface design, Doctoral dissertation, Technische Universität, Berlin

    Google Scholar 

  • Hurtienne J, Blessing L (2007) Design for intuitive use-testing image schema theory for user interface design. In: 16th International Conference on Engineering Design ICED’07, Cite des Sciences et de l’industrie, Paris, 28–31 August 2007

    Google Scholar 

  • Husserl E (1913) Ideas: general introduction to pure phenomenology. Reprinted 2012. Routledge, London

    Google Scholar 

  • Imaz M, Benyon D (2007) Designing with blends: conceptual foundations of human-computer interaction and software engineering methods. MIT Press, Cambridge, MA

    Google Scholar 

  • Intons-Peterson MJ (1992) Components of auditory imagery. In: Reisberg D (ed) Auditory imagery. Lawrence Erlbaum, Hillsdale, pp 45–71

    Google Scholar 

  • Jensenius AR (2014) To gesture or not? An analysis of terminology in nime proceedings 2001–2013. In Proceedings of the 14th international conference on New Interfaces For Musical Expression (NIME 14), Goldsmiths University of London, London 30 June – 4 July 2014

    Google Scholar 

  • Jensenius AR, Godøy RI (2013) Sonifying the shape of human body motion using motiongrams. Empir Musicol Rev 8(2):73–83

    Article  Google Scholar 

  • Johnson M (1987) The body in the mind: the bodily basis of meaning, imagination, and reason. University of Chicago Press, Chicago

    Book  Google Scholar 

  • Johnson M (2008) The meaning of the body: aesthetics of human understanding. University of Chicago Press, Chicago

    Google Scholar 

  • Johnson M (2010) Embodied knowing through art. In: The Routledge companion to research in the arts. Routledge Handbooks, London, pp 141–151

    Google Scholar 

  • Johnson M (2013) II. Identity, Bodily Meaning, and Art. Conscious Lit Arts 32:15

    Google Scholar 

  • Johnson M (2017) Embodied mind, meaning, and reason: how our bodies give rise to understanding. University of Chicago Press, Chicago

    Book  Google Scholar 

  • Johnson M, Rohrer T (2007) We are live creatures: embodiment, American pragmatism, and the cognitive organism. In: Ziemke T, Zlatev J, Frank R (eds) Body, language and mind, Vol. 1: embodiment. mouton de Gruyter, Amsterdam, pp 17–54

    Google Scholar 

  • Johnson ML, Larson S (2003) “Something in the way she moves” -metaphors of musical motion. Metaphor Symb 18(2):63–84

    Article  Google Scholar 

  • Kabisch E, Kuester F, Penny S (2005) Sonic panoramas: experiments with interactive landscape image sonification. In: The proceedings of the 2005 International conference on augmented tele-existence, Christchurch, Dec 2005, pp 156–163

    Google Scholar 

  • Kiefer M, Sim EJ, Herrnberger B, Grothe J, Hoenig K (2008) The sound of concepts: four markers for a link between auditory and conceptual brain systems. J Neurosci 28(47):12224–12230

    Article  Google Scholar 

  • Klemmer SR, Hartmann B, Takayama L (2006) How bodies matter: five themes for interaction design. In: the Proceedings of the 6th Conference on Designing Interactive systems, University Park, PA, June 26–28 June 2006

    Google Scholar 

  • Koffka K (1931) Principles of Gestalt psychology, vol 44. Routledge, London

    Google Scholar 

  • Köhler W (1929) Gestalt psychology. Liveright Publishing Corporation, New York

    Google Scholar 

  • Kramer G, Walker B, Bonebright T, Cook P, Flowers JH, Miner N, Neuhof J (1997) Sonification report: status of the field and research agenda. US National Science Foundation, Alexandria

    Google Scholar 

  • Lakoff G (1987) Fire, and dangerous things: what categories reveal about the mind. University of Chicago Press, Chicago

    Book  Google Scholar 

  • Lakoff G, Johnson M (1980) Metaphors we live by. University of Chicago Press, Chicago

    Google Scholar 

  • Lakoff G, Johnson M (1999) Philosophy in the flesh: the embodied mind and its challenge to western thought. Basic books, New York

    Google Scholar 

  • Larson S (2012) Musical forces: motion, metaphor, and meaning in music. Indiana University Press, Indiana

    Google Scholar 

  • Leman M (2008) Embodied music cognition and mediation technology. MIT Press, Cambridge, MA

    Google Scholar 

  • Macaranas A, Antle AN, Riecke BE (2012) Bridging the gap: attribute and spatial metaphors for tangible interface design. In: the Proceedings of the 6th international conference on tangible, embedded and embodied interaction, Ontario, 19–22 February 2012

    Google Scholar 

  • Magnusson T, Mendieta EH (2007) The acoustic, the digital and the body: a survey on musical instruments. In Proceedings of the 7th international conference on New Interfaces for Musical Expression (NIME 07), New York University, New York, 6–10 June 2007

    Google Scholar 

  • Martinez M, Besold TR, Abdel-Fattah A, Gust H, Schmidt M, Krumnack U, Kühnberger KU (2012) Theory blending as a framework for creativity in systems for general intelligence. Theor Found Artif Gen Intell 4:219–239

    Article  Google Scholar 

  • Mathews M (1991) The radio baton and conductor program, or: pitch, the most important and least expressive part of music. Comput Music J 15(4):37–46

    Article  Google Scholar 

  • Maturana HR, Varela FJ (1987) The tree of knowledge: the biological roots of human understanding. New Science Library/Shambhala Publications, Colorado

    Google Scholar 

  • McGrenere J, Ho W (2000) Affordances: clarifying an evolving concept. In: Proceedings of the graphics interface 2000 conference, Montréal, 15–17 May 2000

    Google Scholar 

  • McPherson A (2012) TouchKeys: capacitive multi-touch sensing on a physical keyboard. A NIME Reader. Springer International Publishing, Cham, pp 419–432

    Google Scholar 

  • Merleau-Ponty M (1968) The visible and the invisible. Followed by working notes. Northwestern University Press, Chicago

    Google Scholar 

  • Meyer LB (1956) Emotion and meaning in music. University of Chicago Press, Chicago

    Google Scholar 

  • Miranda ER, Bonet N, Kirke A (2016) Blyth–Eastbourne–Wembury: Sonification as a compositional tool in electroacoustic music. In: the Proceedings of the 2nd international conference on new musical concepts, Treviso, 5–6 March 2016

    Google Scholar 

  • Nees MA (2009). Internal representations of auditory frequency: behavioral studies of format and malleability by instructions. Dissertation, Georgia Institute of Technology, Georgia

    Google Scholar 

  • Nees MA, Best K (2013). Modality and encoding strategy effects on a verification task with accelerated speech, visual text, and tones. In: Proceedings of the 19th international conference on Auditory Display ICAD 2013, Lodz Poland, 6–9 July 2013

    Google Scholar 

  • Nees MA, Walker BN (2008) Encoding and representation of information in auditory graphs: descriptive reports of listener strategies for understanding data. In: Proceedings of the 14th International Conference on Auditory Display, Paris, France, June 24–27, 2008

    Google Scholar 

  • Nees MA, Walker BN (2011) Auditory displays for in-vehicle technologies. Rev Hum Factors and Ergon 7(1):58–99

    Google Scholar 

  • Neuhoff JG (2004) Ecological psychoacoustics. Elsevier Academic Press, Boston

    Google Scholar 

  • Neuhoff JG (2011) Perception, cognition and action in auditory displays. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 63–85

    Google Scholar 

  • Neuhoff JG, Heller LM (2005) One small step: Sound sources and events as the basis for auditory graphs. 1st Symposium on Auditory Graphs. In: the Proceedings of the 11th International Conference on Auditory Display (ICAD 2005), Limerick, 6–9 July 2005

    Google Scholar 

  • Newell A, Simon H (1976) Computer science as empirical inquiry: symbols and search. Commun ACM 19:113–126

    Article  MathSciNet  Google Scholar 

  • Noë A (2009) Out of our heads: why you are not your brain, and other lessons from the biology of consciousness. Hill & Wang, New York

    Google Scholar 

  • Norman D (1988) The psychology of everyday things. Basic Books, New York

    Google Scholar 

  • Núñez R, Freeman W (1999) Reclaiming cognition: the primacy of action, intention and emotion. Imprint Academic, Exeter

    Google Scholar 

  • O’Modhráin S, Essl G (2004) PebbleBox and CrumbleBag: tactile interfaces for granular synthesis. In: The proceedings of the 4th international conference on New Interfaces for Musical Expression NIME 04, Shizuoka University of Art and Culture Hamamatsu, 3–5 June 2004

    Google Scholar 

  • Paine G (2009) Towards unified design guidelines for new interfaces for musical expression. Org Sound 14(2):142–155

    Article  Google Scholar 

  • Peres SC (2012) A comparison of sound dimensions for auditory graphs: pitch is not so perfect. J Audio Eng Soc 60(7/8):561–567

    Google Scholar 

  • Peres SC, Lane DM (2005) Auditory graphs: the effects of redundant dimensions and divided attention. In: Proceedings of the 11th International Conference on Auditory Display (ICAD2005) Limerick, Ireland, pp 169–174

    Google Scholar 

  • Polotti P, Delle Monache S, Papetti S, Rocchesso D (2008) Gamelunch: forging a dining experience through sound. In: CHI’08 extended abstracts on human factors in computing systems. ACM, New York, pp 2281–2286

    Google Scholar 

  • Putnam H (1967) Psychophysical predicates. In: Capitan W, Merrill D (eds) Art, mind, and religion. University of Pittsburgh Press, Pittsburgh

    Google Scholar 

  • Rimland J, Ballora M, Shumaker W (2013) Beyond visualization of big data: a multi-stage data exploration approach using visualization, sonification, and storification. In: SPIE defense, security, and sensing, international society for optics and photonics, May, pp 87580K–87580K

    Google Scholar 

  • Roads C (1996) The computer music tutorial. MIT Press, Cambridge, MA

    Google Scholar 

  • Rocchesso D, Bresin R (2007) Emerging sounds for disappearing computers. In: Streitz N, Kameas A, Mavrommati I (eds) The disappearing computer. Springer, Berlin/Heidelberg, pp 233–254

    Chapter  Google Scholar 

  • Rocchesso D, Polotti P, Delle Monache S (2009) Designing continuous sonic interaction. Int J Des 3(3):13–25

    Google Scholar 

  • Roddy S (2015) Embodied Sonification. Doctoral thesis, Trinity College Dublin, Dublin, Ireland. Retrieved from http://hdl.handle.net/2262/80558

  • Roddy S, Bridges B (2016) Sounding human with data: the role of embodied conceptual metaphors and aesthetics in representing and exploring data sets. In Music Technology Workshop

    Google Scholar 

  • Roddy S, Furlong D (2013) Rethinking the transmission medium in live computer music performance. Paper presented at the Irish Sound Science and Technology Association (ISSTA) Convocation, Dún Laoghaire Institute of Art, Design and Technology, Dún Laoghaire, Ireland. Available at: http://issta.ie/wpcontent/uploads/ISSTC-2013-RODDY.pdf. Accessd 22 December, 2017

  • Roddy S, Furlong D (2014) Embodied aesthetics in auditory display. Organ Sound 19(1):70–77

    Google Scholar 

  • Ryle G (1949) The concept of mind. Hutchinson, London

    Google Scholar 

  • Scaletti C (1994) Sound synthesis algorithms for auditory data representations. In: Kramer G (ed) Auditory display, sonification, audification and auditory interfaces, SFI studies in the sciences of complexity, proceedings volume XVIII. Addison-Wesley Publishing company, Reading

    Google Scholar 

  • Schaeffer P (1966) Traité des objets musicaux. du Seuil, Paris

    Google Scholar 

  • Schafer RM (1977) The tuning of the world. Alfred A. Knopf, New York

    Google Scholar 

  • Searle JR (1980) Minds, brains, and programs. Behav Brain Sci 3(03):417–424

    Article  Google Scholar 

  • Seibert G, Hug D, Cslovjecsek M (2015) Towards an enactive swimming sonification: exploring multisensory design and musical interpretation. In: Proceedings of the audio mostly 2015 conference: on interaction with sound, Thessaloniki, Greece, October 79 2015, ACM. New York, p 27. doi: https://doi.org/10.1145/2814895.2814902

  • Serafin S, Young D (2004) Toward a generalized friction controller: from the bowed string to unusual musical instruments. In: The proceedings of the 4th international conference on New interfaces for musical expression NIME 04, Shizuoka University of Art and Culture Hamamatsu, 3–5 June 2004

    Google Scholar 

  • Serafin S, Franinovic K, Hermann T, Lemaitre G, Rinott M, Rocchesso D (2011) Sonic interaction design. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 87–110

    Google Scholar 

  • Smalley D (1986) Spectro-morphology and structuring processes. In: Emmerson (ed) The language of electroacoustic music. Palgrave Macmillan, London, pp 61–93

    Chapter  Google Scholar 

  • Smalley D (1997) Spectromorphology: explaining sound-shapes. Org Sound 2(2):107–126

    Article  Google Scholar 

  • Solomon JW (2007) Spatialization in music: the analysis and interpretation of spatial gestures. Dissertation, University of Georgia

    Google Scholar 

  • Steels L (2008) The symbol grounding problem has been solved. So what’s next? Symbols and embodiment: debates on meaning and cognition. OUP USA, New York, pp 223–244

    Google Scholar 

  • Supper A (2012) Lobbying for the ear: the public fascination with and academic legitimacy of the sonification of scientific data. Dissertation, Maastricht University, Maastricht, The Netherlands

    Google Scholar 

  • Tanaka A, Knapp RB (2002) Multimodal interaction in music using the electromyogram and relative position sensing. In: Proceedings of the 2002 international conference on new interfaces for musical expression, Dublin, Ireland, 24–26 May 2002, pp 171–176

    Google Scholar 

  • Todes S (2001) Body and world. MIT Press, Cambridge, MA

    Book  Google Scholar 

  • Truax B (1984) Acoustic communication. Ablex Publishing Corporation, Norwood

    Google Scholar 

  • Truax B (1996) Soundscape, acoustic communication and environmental sound composition. Contemp Music Rev 15(1–2):49–65

    Article  Google Scholar 

  • Tuuri K (2009) Gestural attributions as semantics in user interface sound design. In: International gesture workshop. Springer, Berlin/Heidelberg, pp 257–268

    Google Scholar 

  • Varela FJ, Thompson ET, Rosch E (1991) The embodied mind: cognitive science and human experience. MIT Press, Cambridge, MA

    Book  Google Scholar 

  • Verona D, Peres SC (2017) A comparison between the efficacy of task-based Vs. Data-based sEMG sonification designs. In: Proceedings of the 23rd International Conference on Auditory Display (ICAD 2017), Pennsylvania State University, June 20–23, 2017. URI: http://hdl.handle.net/1853/58380

  • Vickers P (2005) Ars Informatica -- Ars electronica: improving Sonification aesthetics. In: Understanding and designing for aesthetic experience (workshop at HCI 2005: the 19th British HCI group annual conference), 5–9 September 2005, Edinburgh, UK

    Google Scholar 

  • Vickers P (2012) Ways of listening and modes of being: electroacoustic auditory display. J Sonic Stud 2(1):1–23

    Google Scholar 

  • Vickers P, Hogg B (2006) Sonification abstraite/sonification concrete: an ‘aesthetic perspective space’ for classifying auditory displays in the ars musica domain. In Proceedings of the 12th International Conference on Auditory Display (ICAD 2006), London, UK, June 2006

    Google Scholar 

  • Vogt P (2002) The physical symbol grounding problem. Cogn Syst Res 3(3):429–457

    Article  Google Scholar 

  • Wakkary R, Hatala M, Lovell R, Droumeva, M (2005) An ambient intelligence platform for physical play. In Proceedings of the 13th annual ACM International Conference on Multimedia, pp 764–773

    Google Scholar 

  • Walker BN (2002) Magnitude estimation of conceptual data dimensions for use in sonification. J Exp Psychol Appl 8(4):211

    Article  Google Scholar 

  • Walker BN, Kramer G (2004) Ecological psychoacoustics and auditory displays: hearing, grouping, and meaning making. Ecological Psychoacoustics:150–175

    Google Scholar 

  • Walker BN, Nees MA (2011) Chapter 2: theory of sonification. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 9–39

    Google Scholar 

  • Wanderley M, Orio N (2002) Evaluation of input devices for musical expression: borrowing tools from HCI. Comput Music J 26(3):62–76. https://doi.org/10.1162/014892602320582981

    Article  Google Scholar 

  • Wang G (2007) A history of programming and music. In: Collins N (ed) The Cambridge companion to electronic music. Cambridge University Press, Cambridge, pp 55–71

    Chapter  Google Scholar 

  • Waterworth J, Riva G (2014) Feeling present in the physical world and in computer–mediated environments. Palgrave Macmillan, Basingstoke. https://doi.org/10.1057/9781137431677.0001

    Book  Google Scholar 

  • Wertheimer M (1938) Laws of organization in perceptual forms. In: A source book of gestalt psychology. https://doi.org/10.1037/11496-005

    Chapter  Google Scholar 

  • Wessel D, Wright M (2001) Problems and prospects for intimate musical control of computers. In: Proceedings of the international conference on new interfaces for musical expression., 2001, Seattle, WA, 1–2 April, 2001

    Google Scholar 

  • Wilkie K, Holland S, Mulholland P (2010) What can the language of musicians tell us about music interaction design? Comput Music J 34(4):34–48. https://doi.org/10.1162/COMJ_a_00024

    Article  Google Scholar 

  • Wishart T (1996) On sonic art (Contemporary Music Studies Vol 12) Emmerson S Harwood Academic Publishers, Amsterdam

    Google Scholar 

  • Worrall D (2009) Sonification and information: concepts, instruments and techniques. Dissertation: Australian National University, Canberra, Australia

    Google Scholar 

  • Worrall D (2010) Using sound to identify correlations in market data. Auditory Display: the Proceedings of the 6th International Symposium, CMMR/ICAD 2009, Copenhagen, Denmark. Lecture Notes in Computer Science book series, volume 5954, Springer, Heidelberg, pp 202–218

    Google Scholar 

  • Worrall D (2013) Understanding the Need for Micro-Gestural Inflections in Parameter-Mapping Sonification. In Proceedings of the International Conference on Auditory Display (ICAD), Lodz, Poland. http://hdl.handle.net/1853/51668

  • Worrall D (2014) Can micro-gestural inflections be used to improve the soniculatory effectiveness of parameter mapping sonifications? Org Sound 19(1):52–59. https://doi.org/10.1017/S135577181300040X

    Article  Google Scholar 

  • Zbikowski LM (2005) Conceptualizing music: cognitive structure, theory, and analysis. Oxford University Press, New York

    Google Scholar 

Download references

Acknowledgements

This publication has been funded by an Irish Research Council Government of Ireland Postdoctoral Fellwoship Award. This publication has emanated from research supported in part by a research grant from Science Foundation Ireland (SFI) and is co-funded under the European Regional Development Fund under Grant Number 13/RC/2077.

Author information

Authors and Affiliations

  1. Connect Centre, Trinity College Dublin, Dublin, Republic of Ireland

    Stephen Roddy

  2. Ulster University, Coleraine, Republic of Ireland

    Brian Bridges

Authors
  1. Stephen Roddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian Bridges
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen Roddy .

Editor information

Editors and Affiliations

  1. School of Interactive Arts & Technology, Simon Fraser University, Surrey, BC, Canada

    Michael Filimowicz

  2. School of Communications, Simon Fraser University, Burnaby, BC, Canada

    Veronika Tzankova

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Roddy, S., Bridges, B. (2018). Sound, Ecological Affordances and Embodied Mappings in Auditory Display. In: Filimowicz, M., Tzankova, V. (eds) New Directions in Third Wave Human-Computer Interaction: Volume 2 - Methodologies . Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-73374-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-73374-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-73373-9

  • Online ISBN: 978-3-319-73374-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation