Sound, Ecological Affordances and Embodied Mappings in Auditory Display

Part of the Human–Computer Interaction Series book series (HCIS)


The third wave of HCI has seen the widespread adoption of design principles borrowed from and informed by breakthroughs in the field of embodied cognitive science. These developments have taken place primarily in the contexts of visual media and interaction, but they are also of importance to the design of auditory displays and interactive systems in which sounds plays a dominant role, where they open up new affordances by which information might be communicated to a listener. This chapter examines the relationship between auditory display, sonic interaction design and embodied cognition and explores frameworks from embodied cognition that might inform the design of more informative auditory displays in a variety of contexts. It will do so by addressing these issues from an interdisciplinary perspective, bringing together insights from cognitive science and philosophy, general HCI and computer science, along with music theory and practice.



This publication has been funded by an Irish Research Council Government of Ireland Postdoctoral Fellwoship Award. This publication has emanated from research supported in part by a research grant from Science Foundation Ireland (SFI) and is co-funded under the European Regional Development Fund under Grant Number 13/RC/2077.


  1. Antle AN, Corness G, Bevans A (2011) Springboard: designing embodied schema based embodied interaction for an abstract domain. In: England D (ed) Human-computer interaction series: whole body interaction, Springer, Heidelberg, pp 7–18CrossRefGoogle Scholar
  2. Armstrong DF, Stokoe WC, Wilcox SE (1995) Gesture and the nature of language. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  3. Arnheim R (1954) Art and visual perception: a psychology of the creative eye. University of California Press, Los AngelesGoogle Scholar
  4. Arnheim R (1984) Perceptual dynamics in musical expression. Music Q 70(3):295–309CrossRefGoogle Scholar
  5. Bannon L (1991) From human factors to human actors: the role of psychology and human-computer interaction studies in system design. Des Work Coop Des Comp Systems 25:44Google Scholar
  6. Bannon LJ, Bødker S (1989) Beyond the interface: encountering artifacts in use. DAIMI Rep Ser 18:288Google Scholar
  7. Barass S, Jeon M, Fernström M (2018) Editorial introduction in ergonomics in design special issue: sonic information design (forthcoming)Google Scholar
  8. Barrass S (1998) Auditory information design. Dissertation, Australian National University, CanberraGoogle Scholar
  9. Barrass S (2012) The aesthetic turn in sonification towards a social and cultural medium. AI & Soc 27(2):177–181CrossRefGoogle Scholar
  10. Barrass S, Vickers P (2011) Sonification design and aesthetics. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 145–171Google Scholar
  11. Barrett N (2015) Creating tangible spatial-musical images from physical performance gestures. In: Proceedings of the 2015 Conference on New Interfaces for Musical Expression (NIME 15), Louisiana State University, Baton Rouge, 31 May–3 June 2015Google Scholar
  12. Barsalou LW (2010) Grounded cognition: past, present, and future. Top Cogn Sci 2(4):716–724CrossRefGoogle Scholar
  13. Bødker S (2015) Third-wave HCI, 10 years later---participation and sharing. Interactions 22(5):24–31CrossRefGoogle Scholar
  14. Bødker S, Klokmose CN (2016) Dynamics, multiplicity and conceptual blends in HCI. In: Proceedings of the 2016 CHI conference on human factors in computing systems, San Jose, CA, 7–12 May 2016Google Scholar
  15. Bovermann T, Hermann T, Ritter H (2006) A tangible environment for ambient data representation. In: Proceedings of the first international workshop on haptic and audio interaction design, Glasgow, 31 August - 1 September 2006Google Scholar
  16. Brandt PA (2004) Spaces, domains, and meaning: essays in cognitive semiotics. In: Bax M, Van Heusden B, Wildgen W, Brandt PA (eds) European semiotics: language, cognition, and culture. 4, Peter Lang Publishing, BernGoogle Scholar
  17. Brazil E, Fernström M (2006) Investigating concurrent auditory icon recognition. In: Proceedings of the 12th International Conference on Auditory Display (ICAD 2006), London, 20–23 June 2006Google Scholar
  18. Bregman AS (1990) Auditory scene analysis. MIT Press, Cambridge, MAGoogle Scholar
  19. Bregman AS (1994) Auditory scene analysis: the perceptual organization of sound. In: McAdams S, Bigand E (eds) Thinking in sound: the cognitive psychology of human audition, vol 95. Oxford University Press, New York, p 1177Google Scholar
  20. Brooks R (2003) Robot: the future of flesh and machines. Penguin Books, LondonGoogle Scholar
  21. Brower C (2000) A cognitive theory of musical meaning. J Music Theory 44(2):323–379CrossRefGoogle Scholar
  22. Brown B, Aaron M (2001) The politics of nature. In: Smith J (ed) The rise of modern genomics, 3rd edn. Wiley, New York, pp 234–295Google Scholar
  23. Buxton WAS (1987) The haptic channel. In: Baecker RM, Buxton WAS (eds) Readings in human computer interaction: a multidisciplinary approach. Morgan Kaufmann, San Mateo, pp 357–365Google Scholar
  24. Card S, Moran T, Newell A (1986) The model human processor- an engineering model of human performance. Handb Percept Hum Perform 2:45–41Google Scholar
  25. Cook P (2001) Principles for designing computer music controllers. In: Proceedings of the 2001 conference on new interfaces for musical expression (NIME 2001), Seattle, WA, 1–2 April 2001Google Scholar
  26. Cox A (2001) The mimetic hypothesis and embodied musical meaning. Music Sci 5(2):195–212CrossRefGoogle Scholar
  27. Deacon T (2006) The aesthetic faculty. In: Turner M (ed) The artful mind: cognitive science and the riddle of human creativity. Oxford University Press, OxfordGoogle Scholar
  28. Dewey J (1934) Art as experience. Minton, Balch and Co., New YorkGoogle Scholar
  29. DeWitt A, Bresin R (2007) Sound design for affective interaction. In: International Conference on Affective Computing and Intelligent Interaction. Springer, Heidelberg, pp 523–533CrossRefGoogle Scholar
  30. Diniz N (2012) A multimodal framework for interactive sonification and sound-based communication. Dissertation, Ghent University, BelgiumGoogle Scholar
  31. Diniz N, Deweppe A, Demey M Leman M (2010) A Framework for music-based interactive sonification. In: the Proceedings of the 16th International Conference on Auditory Display (ICAD 2010), Washington DC, 9–15 June 2010Google Scholar
  32. Diniz N, Coussement P, Deweppe A, Demey M, Leman M (2012) An embodied music cognition approach to multilevel interactive sonification. J Multimodal User Interfaces 5(3–4):211–219CrossRefGoogle Scholar
  33. Dourish P (2004) Where the action is: the foundations of embodied interaction. MIT Press, Cambridge, MAGoogle Scholar
  34. Dreyfus HL (1965) Alchemy and artificial intelligence. The Rand Corporation, Research Report, Santa MonicaGoogle Scholar
  35. Droumeva M, de Castell S, Wakkary R (2007) Investigating sound intensity gradients as feedback for embodied learning. In: The Proceedings of the 13th International Conference for Auditory Display (ICAD 2013), Montreal, 26–29 June, 2007Google Scholar
  36. Droumeva M, Wakkary R (2008) Understanding aural fluency in auditory display design for ambient intelligent environments. In Proceedings of the 14th International Conference on Auditory Displays, Paris, France, June 24–27, 2008Google Scholar
  37. Faste T, Faste H (2012) Demystifying “design research”: Design is not research, research is design. In: The Proceedings of the industrial designers society of america education symposium, Boston, 15 August 2012Google Scholar
  38. Fauconnier G, Turner M (2002) The way we think: conceptual blending and the mind’s hidden complexities. Basic Books, New YorkGoogle Scholar
  39. Fencott R, Bryan-Kinns N (2010, June) Hey Man, You’re Invading my Personal Space! Privacy and Awareness in Collaborative Music. In NIME (pp. 198–203)Google Scholar
  40. Flowers JH (2005) Thirteen years of reflection on auditory graphing: promises, pitfalls, and potential new directions. In: The proceedings of the 11th International Conference on Auditory Display (ICAD 2005), Limerick, 6–9 July 2005Google Scholar
  41. Fodor J (1975) The language of thought. Thomas Y. Crowell, New YorkGoogle Scholar
  42. Gardner H (1985) The mind’s new science. Basic Books, New YorkGoogle Scholar
  43. Gaver WW (1989) The SonicFinder: an interface that uses auditory icons. Hum Comput Interaction 4(1):67–94CrossRefGoogle Scholar
  44. Gibson JJ (1977) The theory of affordances. In: Shaw R, Bransford J (eds) Perceiving, acting, and knowing: toward an ecological psychology. Lawrence Erlbaum, Hillsdale, pp 67–82Google Scholar
  45. Gibson JJ (1978) The ecological approach to the visual perception of pictures. Leonardo 11(3):227–235CrossRefGoogle Scholar
  46. Glenberg AM, Robertson DA (2000) Symbol grounding and meaning: a comparison of high-dimensional and embodied theories of meaning. J Mem Lang 43(3):379–401CrossRefGoogle Scholar
  47. Godøy RI (1997) Knowledge in music theory by shapes of musical objects and sound-producing actions. In: Leman M (ed) Music, gestalt, and computing. Springer, Berlin/Heidelberg, pp 89–102CrossRefGoogle Scholar
  48. Godøy RI (2003) Motor-mimetic music cognition. Leonardo 36(4):317–319CrossRefGoogle Scholar
  49. Godøy RI (2006) Gestural-sonorous objects: embodied extensions of Schaeffer’s conceptual apparatus. Org Sound 11(02):149–157CrossRefGoogle Scholar
  50. Gossmann J (2010) From metaphor to medium: sonification as extension of our body In: Proceedings of the16th International Conference on Auditory Display (ICAD2010), Washington, DC, June 9–15, 2010Google Scholar
  51. Graham R, Bridges B (2014) Gesture and embodied metaphor in spatial music performance Systems Design. In: Proceedings of the 2014 conference on New Interfaces for Musical Expression (NIME 14), Goldsmiths University of London, London 30 June - 3 July 2014Google Scholar
  52. Graham R, Bridges B (2015) Managing musical complexity with embodied metaphors. In: Proceedings of the 2015 conference on New Interfaces for Musical Expression (NIME 15), Louisiana State University, Baton Rouge, 31 May–3 June 2015Google Scholar
  53. Graham R, Manzione C, Bridges B, Brent W (2017) Exploring Pitch and Timbre through 3D spaces: embodied models in virtual reality as a basis for performance systems design. In: Proceedings of the 2017 conference on New Interfaces for Musical Expression (NIME 17), Copenhagen, 15 May–19 May 2017Google Scholar
  54. Grey JM (1977) Multidimensional perceptual scaling of musical timbres. J Acoust Soc Am 61(5):1270–1277CrossRefGoogle Scholar
  55. Grond F (2013) Listening–Mode–Centered Sonification Design for Data Exploration. Dissertation, Technische Fakultät der Universität Bielefeld, BielefeldGoogle Scholar
  56. Grond F, Berger J (2011) Parameter mapping sonification. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 363–397Google Scholar
  57. Gurevich M, Treviño J (2007) Expression and its discontents: toward an ecology of musical creation. In Proceedings of the 7th International Conference on New Interfaces for Musical Expression (NIME 2007), New York University, New York, 6–10 June 2007Google Scholar
  58. Halpern AR, Zatorre RJ, Bouffard M, Johnson JA (2004) Behavioral and neural correlates of perceived and imagined musical timbre. Neuropsychologia 42(9):1281–1292CrossRefGoogle Scholar
  59. Hampe B (2005) Image schemas in cognitive linguistics: introduction. In: Hampe B, Grady JE (eds) From perception to meaning: image schemas in cognitive linguistics. Walter de Gruyter, BerlinCrossRefGoogle Scholar
  60. Harnad S (1990) The symbol grounding problem. Physica D: Nonlinear Phenom 42(1):335–346CrossRefGoogle Scholar
  61. Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. In the Proceedings of Alt. Chi. Session at the SIGCHI conference on human factors in computing systems, San Jose, 28 April – 3 May 2007Google Scholar
  62. Heidegger M (1927) The basic problems of phenomenology. Indiana University Press, BloomingtonGoogle Scholar
  63. Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: The proceedings of the 14th International Conference on Auditory Display (ICAD 2008), IRCAM Paris, 24–27 June 2008Google Scholar
  64. Hubbard TL (2010) Auditory imagery: empirical findings. Psychol Bull 136(2):302CrossRefGoogle Scholar
  65. Hunt A, Wanderley MM, Paradis M (2002) The importance of parameter mapping in electronic instrument design. J New Music Res 32(4):429–440CrossRefGoogle Scholar
  66. Hurtienne J (2009) Image schemas and design for intuitive use: exploring new guidance for user interface design, Doctoral dissertation, Technische Universität, BerlinGoogle Scholar
  67. Hurtienne J, Blessing L (2007) Design for intuitive use-testing image schema theory for user interface design. In: 16th International Conference on Engineering Design ICED’07, Cite des Sciences et de l’industrie, Paris, 28–31 August 2007Google Scholar
  68. Husserl E (1913) Ideas: general introduction to pure phenomenology. Reprinted 2012. Routledge, LondonGoogle Scholar
  69. Imaz M, Benyon D (2007) Designing with blends: conceptual foundations of human-computer interaction and software engineering methods. MIT Press, Cambridge, MAGoogle Scholar
  70. Intons-Peterson MJ (1992) Components of auditory imagery. In: Reisberg D (ed) Auditory imagery. Lawrence Erlbaum, Hillsdale, pp 45–71Google Scholar
  71. Jensenius AR (2014) To gesture or not? An analysis of terminology in nime proceedings 2001–2013. In Proceedings of the 14th international conference on New Interfaces For Musical Expression (NIME 14), Goldsmiths University of London, London 30 June – 4 July 2014Google Scholar
  72. Jensenius AR, Godøy RI (2013) Sonifying the shape of human body motion using motiongrams. Empir Musicol Rev 8(2):73–83CrossRefGoogle Scholar
  73. Johnson M (1987) The body in the mind: the bodily basis of meaning, imagination, and reason. University of Chicago Press, ChicagoGoogle Scholar
  74. Johnson M (2008) The meaning of the body: aesthetics of human understanding. University of Chicago Press, ChicagoGoogle Scholar
  75. Johnson M (2010) Embodied knowing through art. In: The Routledge companion to research in the arts. Routledge Handbooks, London, pp 141–151Google Scholar
  76. Johnson M (2013) II. Identity, Bodily Meaning, and Art. Conscious Lit Arts 32:15Google Scholar
  77. Johnson M (2017) Embodied mind, meaning, and reason: how our bodies give rise to understanding. University of Chicago Press, ChicagoCrossRefGoogle Scholar
  78. Johnson M, Rohrer T (2007) We are live creatures: embodiment, American pragmatism, and the cognitive organism. In: Ziemke T, Zlatev J, Frank R (eds) Body, language and mind, Vol. 1: embodiment. mouton de Gruyter, Amsterdam, pp 17–54Google Scholar
  79. Johnson ML, Larson S (2003) “Something in the way she moves” -metaphors of musical motion. Metaphor Symb 18(2):63–84CrossRefGoogle Scholar
  80. Kabisch E, Kuester F, Penny S (2005) Sonic panoramas: experiments with interactive landscape image sonification. In: The proceedings of the 2005 International conference on augmented tele-existence, Christchurch, Dec 2005, pp 156–163Google Scholar
  81. Kiefer M, Sim EJ, Herrnberger B, Grothe J, Hoenig K (2008) The sound of concepts: four markers for a link between auditory and conceptual brain systems. J Neurosci 28(47):12224–12230CrossRefGoogle Scholar
  82. Klemmer SR, Hartmann B, Takayama L (2006) How bodies matter: five themes for interaction design. In: the Proceedings of the 6th Conference on Designing Interactive systems, University Park, PA, June 26–28 June 2006Google Scholar
  83. Koffka K (1931) Principles of Gestalt psychology, vol 44. Routledge, LondonGoogle Scholar
  84. Köhler W (1929) Gestalt psychology. Liveright Publishing Corporation, New YorkGoogle Scholar
  85. Kramer G, Walker B, Bonebright T, Cook P, Flowers JH, Miner N, Neuhof J (1997) Sonification report: status of the field and research agenda. US National Science Foundation, AlexandriaGoogle Scholar
  86. Lakoff G (1987) Fire, and dangerous things: what categories reveal about the mind. University of Chicago Press, ChicagoCrossRefGoogle Scholar
  87. Lakoff G, Johnson M (1980) Metaphors we live by. University of Chicago Press, ChicagoGoogle Scholar
  88. Lakoff G, Johnson M (1999) Philosophy in the flesh: the embodied mind and its challenge to western thought. Basic books, New YorkGoogle Scholar
  89. Larson S (2012) Musical forces: motion, metaphor, and meaning in music. Indiana University Press, IndianaGoogle Scholar
  90. Leman M (2008) Embodied music cognition and mediation technology. MIT Press, Cambridge, MAGoogle Scholar
  91. Macaranas A, Antle AN, Riecke BE (2012) Bridging the gap: attribute and spatial metaphors for tangible interface design. In: the Proceedings of the 6th international conference on tangible, embedded and embodied interaction, Ontario, 19–22 February 2012Google Scholar
  92. Magnusson T, Mendieta EH (2007) The acoustic, the digital and the body: a survey on musical instruments. In Proceedings of the 7th international conference on New Interfaces for Musical Expression (NIME 07), New York University, New York, 6–10 June 2007Google Scholar
  93. Martinez M, Besold TR, Abdel-Fattah A, Gust H, Schmidt M, Krumnack U, Kühnberger KU (2012) Theory blending as a framework for creativity in systems for general intelligence. Theor Found Artif Gen Intell 4:219–239CrossRefGoogle Scholar
  94. Mathews M (1991) The radio baton and conductor program, or: pitch, the most important and least expressive part of music. Comput Music J 15(4):37–46CrossRefGoogle Scholar
  95. Maturana HR, Varela FJ (1987) The tree of knowledge: the biological roots of human understanding. New Science Library/Shambhala Publications, ColoradoGoogle Scholar
  96. McGrenere J, Ho W (2000) Affordances: clarifying an evolving concept. In: Proceedings of the graphics interface 2000 conference, Montréal, 15–17 May 2000Google Scholar
  97. McPherson A (2012) TouchKeys: capacitive multi-touch sensing on a physical keyboard. A NIME Reader. Springer International Publishing, Cham, pp 419–432CrossRefGoogle Scholar
  98. Merleau-Ponty M (1968) The visible and the invisible. Followed by working notes. Northwestern University Press, ChicagoGoogle Scholar
  99. Meyer LB (1956) Emotion and meaning in music. University of Chicago Press, ChicagoGoogle Scholar
  100. Miranda ER, Bonet N, Kirke A (2016) Blyth–Eastbourne–Wembury: Sonification as a compositional tool in electroacoustic music. In: the Proceedings of the 2nd international conference on new musical concepts, Treviso, 5–6 March 2016Google Scholar
  101. Nees MA (2009). Internal representations of auditory frequency: behavioral studies of format and malleability by instructions. Dissertation, Georgia Institute of Technology, GeorgiaGoogle Scholar
  102. Nees MA, Best K (2013). Modality and encoding strategy effects on a verification task with accelerated speech, visual text, and tones. In: Proceedings of the 19th international conference on Auditory Display ICAD 2013, Lodz Poland, 6–9 July 2013Google Scholar
  103. Nees MA, Walker BN (2008) Encoding and representation of information in auditory graphs: descriptive reports of listener strategies for understanding data. In: Proceedings of the 14th International Conference on Auditory Display, Paris, France, June 24–27, 2008Google Scholar
  104. Nees MA, Walker BN (2011) Auditory displays for in-vehicle technologies. Rev Hum Factors and Ergon 7(1):58–99CrossRefGoogle Scholar
  105. Neuhoff JG (2004) Ecological psychoacoustics. Elsevier Academic Press, BostonGoogle Scholar
  106. Neuhoff JG (2011) Perception, cognition and action in auditory displays. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 63–85Google Scholar
  107. Neuhoff JG, Heller LM (2005) One small step: Sound sources and events as the basis for auditory graphs. 1st Symposium on Auditory Graphs. In: the Proceedings of the 11th International Conference on Auditory Display (ICAD 2005), Limerick, 6–9 July 2005Google Scholar
  108. Newell A, Simon H (1976) Computer science as empirical inquiry: symbols and search. Commun ACM 19:113–126MathSciNetCrossRefGoogle Scholar
  109. Noë A (2009) Out of our heads: why you are not your brain, and other lessons from the biology of consciousness. Hill & Wang, New YorkGoogle Scholar
  110. Norman D (1988) The psychology of everyday things. Basic Books, New YorkGoogle Scholar
  111. Núñez R, Freeman W (1999) Reclaiming cognition: the primacy of action, intention and emotion. Imprint Academic, ExeterGoogle Scholar
  112. O’Modhráin S, Essl G (2004) PebbleBox and CrumbleBag: tactile interfaces for granular synthesis. In: The proceedings of the 4th international conference on New Interfaces for Musical Expression NIME 04, Shizuoka University of Art and Culture Hamamatsu, 3–5 June 2004Google Scholar
  113. Paine G (2009) Towards unified design guidelines for new interfaces for musical expression. Org Sound 14(2):142–155CrossRefGoogle Scholar
  114. Peres SC (2012) A comparison of sound dimensions for auditory graphs: pitch is not so perfect. J Audio Eng Soc 60(7/8):561–567Google Scholar
  115. Peres SC, Lane DM (2005) Auditory graphs: the effects of redundant dimensions and divided attention. In: Proceedings of the 11th International Conference on Auditory Display (ICAD2005) Limerick, Ireland, pp 169–174Google Scholar
  116. Polotti P, Delle Monache S, Papetti S, Rocchesso D (2008) Gamelunch: forging a dining experience through sound. In: CHI’08 extended abstracts on human factors in computing systems. ACM, New York, pp 2281–2286Google Scholar
  117. Putnam H (1967) Psychophysical predicates. In: Capitan W, Merrill D (eds) Art, mind, and religion. University of Pittsburgh Press, PittsburghGoogle Scholar
  118. Rimland J, Ballora M, Shumaker W (2013) Beyond visualization of big data: a multi-stage data exploration approach using visualization, sonification, and storification. In: SPIE defense, security, and sensing, international society for optics and photonics, May, pp 87580K–87580KGoogle Scholar
  119. Roads C (1996) The computer music tutorial. MIT Press, Cambridge, MAGoogle Scholar
  120. Rocchesso D, Bresin R (2007) Emerging sounds for disappearing computers. In: Streitz N, Kameas A, Mavrommati I (eds) The disappearing computer. Springer, Berlin/Heidelberg, pp 233–254CrossRefGoogle Scholar
  121. Rocchesso D, Polotti P, Delle Monache S (2009) Designing continuous sonic interaction. Int J Des 3(3):13–25Google Scholar
  122. Roddy S (2015) Embodied Sonification. Doctoral thesis, Trinity College Dublin, Dublin, Ireland. Retrieved from
  123. Roddy S, Bridges B (2016) Sounding human with data: the role of embodied conceptual metaphors and aesthetics in representing and exploring data sets. In Music Technology WorkshopGoogle Scholar
  124. Roddy S, Furlong D (2013) Rethinking the transmission medium in live computer music performance. Paper presented at the Irish Sound Science and Technology Association (ISSTA) Convocation, Dún Laoghaire Institute of Art, Design and Technology, Dún Laoghaire, Ireland. Available at: Accessd 22 December, 2017
  125. Roddy S, Furlong D (2014) Embodied aesthetics in auditory display. Organ Sound 19(1):70–77CrossRefGoogle Scholar
  126. Ryle G (1949) The concept of mind. Hutchinson, LondonGoogle Scholar
  127. Scaletti C (1994) Sound synthesis algorithms for auditory data representations. In: Kramer G (ed) Auditory display, sonification, audification and auditory interfaces, SFI studies in the sciences of complexity, proceedings volume XVIII. Addison-Wesley Publishing company, ReadingGoogle Scholar
  128. Schaeffer P (1966) Traité des objets musicaux. du Seuil, ParisGoogle Scholar
  129. Schafer RM (1977) The tuning of the world. Alfred A. Knopf, New YorkGoogle Scholar
  130. Searle JR (1980) Minds, brains, and programs. Behav Brain Sci 3(03):417–424CrossRefGoogle Scholar
  131. Seibert G, Hug D, Cslovjecsek M (2015) Towards an enactive swimming sonification: exploring multisensory design and musical interpretation. In: Proceedings of the audio mostly 2015 conference: on interaction with sound, Thessaloniki, Greece, October 79 2015, ACM. New York, p 27. doi:
  132. Serafin S, Young D (2004) Toward a generalized friction controller: from the bowed string to unusual musical instruments. In: The proceedings of the 4th international conference on New interfaces for musical expression NIME 04, Shizuoka University of Art and Culture Hamamatsu, 3–5 June 2004Google Scholar
  133. Serafin S, Franinovic K, Hermann T, Lemaitre G, Rinott M, Rocchesso D (2011) Sonic interaction design. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 87–110Google Scholar
  134. Smalley D (1986) Spectro-morphology and structuring processes. In: Emmerson (ed) The language of electroacoustic music. Palgrave Macmillan, London, pp 61–93CrossRefGoogle Scholar
  135. Smalley D (1997) Spectromorphology: explaining sound-shapes. Org Sound 2(2):107–126CrossRefGoogle Scholar
  136. Solomon JW (2007) Spatialization in music: the analysis and interpretation of spatial gestures. Dissertation, University of GeorgiaGoogle Scholar
  137. Steels L (2008) The symbol grounding problem has been solved. So what’s next? Symbols and embodiment: debates on meaning and cognition. OUP USA, New York, pp 223–244Google Scholar
  138. Supper A (2012) Lobbying for the ear: the public fascination with and academic legitimacy of the sonification of scientific data. Dissertation, Maastricht University, Maastricht, The NetherlandsGoogle Scholar
  139. Tanaka A, Knapp RB (2002) Multimodal interaction in music using the electromyogram and relative position sensing. In: Proceedings of the 2002 international conference on new interfaces for musical expression, Dublin, Ireland, 24–26 May 2002, pp 171–176Google Scholar
  140. Todes S (2001) Body and world. MIT Press, Cambridge, MAGoogle Scholar
  141. Truax B (1984) Acoustic communication. Ablex Publishing Corporation, NorwoodGoogle Scholar
  142. Truax B (1996) Soundscape, acoustic communication and environmental sound composition. Contemp Music Rev 15(1–2):49–65CrossRefGoogle Scholar
  143. Tuuri K (2009) Gestural attributions as semantics in user interface sound design. In: International gesture workshop. Springer, Berlin/Heidelberg, pp 257–268Google Scholar
  144. Varela FJ, Thompson ET, Rosch E (1991) The embodied mind: cognitive science and human experience. MIT Press, Cambridge, MAGoogle Scholar
  145. Verona D, Peres SC (2017) A comparison between the efficacy of task-based Vs. Data-based sEMG sonification designs. In: Proceedings of the 23rd International Conference on Auditory Display (ICAD 2017), Pennsylvania State University, June 20–23, 2017. URI:
  146. Vickers P (2005) Ars Informatica -- Ars electronica: improving Sonification aesthetics. In: Understanding and designing for aesthetic experience (workshop at HCI 2005: the 19th British HCI group annual conference), 5–9 September 2005, Edinburgh, UKGoogle Scholar
  147. Vickers P (2012) Ways of listening and modes of being: electroacoustic auditory display. J Sonic Stud 2(1):1–23MathSciNetGoogle Scholar
  148. Vickers P, Hogg B (2006) Sonification abstraite/sonification concrete: an ‘aesthetic perspective space’ for classifying auditory displays in the ars musica domain. In Proceedings of the 12th International Conference on Auditory Display (ICAD 2006), London, UK, June 2006Google Scholar
  149. Vogt P (2002) The physical symbol grounding problem. Cogn Syst Res 3(3):429–457MathSciNetCrossRefGoogle Scholar
  150. Wakkary R, Hatala M, Lovell R, Droumeva, M (2005) An ambient intelligence platform for physical play. In Proceedings of the 13th annual ACM International Conference on Multimedia, pp 764–773Google Scholar
  151. Walker BN (2002) Magnitude estimation of conceptual data dimensions for use in sonification. J Exp Psychol Appl 8(4):211CrossRefGoogle Scholar
  152. Walker BN, Kramer G (2004) Ecological psychoacoustics and auditory displays: hearing, grouping, and meaning making. Ecological Psychoacoustics:150–175Google Scholar
  153. Walker BN, Nees MA (2011) Chapter 2: theory of sonification. In: Hermann T, Hunt A, Neuhoff JG (eds) The sonification handbook. Logos Publishing House, Berlin, pp 9–39Google Scholar
  154. Wanderley M, Orio N (2002) Evaluation of input devices for musical expression: borrowing tools from HCI. Comput Music J 26(3):62–76. CrossRefGoogle Scholar
  155. Wang G (2007) A history of programming and music. In: Collins N (ed) The Cambridge companion to electronic music. Cambridge University Press, Cambridge, pp 55–71CrossRefGoogle Scholar
  156. Waterworth J, Riva G (2014) Feeling present in the physical world and in computer–mediated environments. Palgrave Macmillan, Basingstoke. CrossRefGoogle Scholar
  157. Wertheimer M (1938) Laws of organization in perceptual forms. In: A source book of gestalt psychology. CrossRefGoogle Scholar
  158. Wessel D, Wright M (2001) Problems and prospects for intimate musical control of computers. In: Proceedings of the international conference on new interfaces for musical expression., 2001, Seattle, WA, 1–2 April, 2001Google Scholar
  159. Wilkie K, Holland S, Mulholland P (2010) What can the language of musicians tell us about music interaction design? Comput Music J 34(4):34–48. CrossRefGoogle Scholar
  160. Wishart T (1996) On sonic art (Contemporary Music Studies Vol 12) Emmerson S Harwood Academic Publishers, AmsterdamGoogle Scholar
  161. Worrall D (2009) Sonification and information: concepts, instruments and techniques. Dissertation: Australian National University, Canberra, AustraliaGoogle Scholar
  162. Worrall D (2010) Using sound to identify correlations in market data. Auditory Display: the Proceedings of the 6th International Symposium, CMMR/ICAD 2009, Copenhagen, Denmark. Lecture Notes in Computer Science book series, volume 5954, Springer, Heidelberg, pp 202–218CrossRefGoogle Scholar
  163. Worrall D (2013) Understanding the Need for Micro-Gestural Inflections in Parameter-Mapping Sonification. In Proceedings of the International Conference on Auditory Display (ICAD), Lodz, Poland.
  164. Worrall D (2014) Can micro-gestural inflections be used to improve the soniculatory effectiveness of parameter mapping sonifications? Org Sound 19(1):52–59. CrossRefGoogle Scholar
  165. Zbikowski LM (2005) Conceptualizing music: cognitive structure, theory, and analysis. Oxford University Press, New YorkGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Connect Centre, Trinity College DublinDublinRepublic of Ireland
  2. 2.Ulster UniversityColeraineRepublic of Ireland

Personalised recommendations