Navigation of interactive sonifications and visualisations of time-series data using multi-touch computing

  • Sam Ferguson
  • Kirsty Beilharz
  • Claudia A. Calò
Original Paper


This paper discusses interaction design for interactive sonification and visualisation of data in multi-touch contexts. Interaction design for data analysis is becoming increasingly important as data becomes more openly available. We discuss how navigation issues such as zooming, selection, arrangement and playback of data relate to both the auditory and visual modality in different ways, and how they may be linked through the modality of touch and gestural interaction. For this purpose we introduce a user interface for exploring and interacting with representations of time-series data simultaneously in both the visual and auditory modalities.


Interaction Sonification Tabletop Data analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adcock M, Barrass S (2004) Cultivating design patterns for auditory displays. In: Proceedings of the 10th international conference on auditory display, Sydney Google Scholar
  2. 2.
    Burkhard R et al. (2007) Visualisation summit 2007: Ten research goals for 2010. Inform Visual 6(3):169–188 MathSciNetCrossRefGoogle Scholar
  3. 3.
    Aldersey-Williams H, Hall P, Sargent T, Antonelli P (2008) Design and the elastic mind. The Museum of Modern Art, New York Google Scholar
  4. 4.
    Anderson E (1935) The irises of the gaspe peninsula. Bull Am Iris Soc 59:2–5 Google Scholar
  5. 5.
    Barrass S (2003) Sonification design patterns. In: Proceedings of the 9th international conference on auditory display, Boston Google Scholar
  6. 6.
    Barrass S, Schaffert N, Barrass T (2010) Probing preferences between six designs of interactive sonifications for recreational sports, health and fitness. In: Proceedings of ISon 2010, 3rd interactive sonification workshop, Stockholm, Sweden Google Scholar
  7. 7.
    Beilharz K, Ferguson S (2009) An interface design for aesthetic and interactive sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark Google Scholar
  8. 8.
    Bertelsen O, Pold S (2004) Criticism as an approach to interface aesthetics. In: Nordic conference on human-computer interaction, Tampere, Finland Google Scholar
  9. 9.
    Bovermann T, Groten J, de Campo A, Eckel G (2007) Juggling sounds. In: International workshop on interactive sonification, York, UK Google Scholar
  10. 10.
    Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. In: Proceedings of the 12th international conference on auditory display, London, UK Google Scholar
  11. 11.
    Chen C (2005) Top 10 unsolved information visualisation problems. IEEE Comput Graph Appl 25(4):12–16 CrossRefGoogle Scholar
  12. 12.
    Collins A, Apted T, Kay J (2007) Tabletop file system access: associative and hierarchical approaches. In: IEEE international workshop on horizontal interactive human-computer systems (TABLETOP’07), pp 113–120 CrossRefGoogle Scholar
  13. 13.
    Collins A, Kay J (2008) Collaborative personal information management with shared, interactive tabletops. In: CHI 2008, Florence, Italy Google Scholar
  14. 14.
    Cowie A (2010) Collaborative concept mapping at an interactive tabletop. PhD thesis Google Scholar
  15. 15.
    Ferguson S (2009) Exploratory sound analysis: statistical sonifications for the investigation of sound. PhD thesis Google Scholar
  16. 16.
    Flowers JH, Buhman DC, Turnage KD (1997) Cross-modal equivalence of visual and auditory scatterplots for exploring bivariate data samples. Hum Factors 39(3):341–351 CrossRefGoogle Scholar
  17. 17.
    Fry B (2004) Computational information design. PhD thesis Google Scholar
  18. 18.
    Fry B (2008) Nature hapmap cover. Nature 437(7063):1 (cover) Google Scholar
  19. 19.
    Geiselhart K (2010) Towards global governance—interactive technologies and global accountability measures. Public Commun Rev 1(1):37–46 Google Scholar
  20. 20.
    Hermann T (2002) Sonification for exploratory data analysis. PhD thesis Google Scholar
  21. 21.
    Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 2008 international conference on auditory display, Paris, France Google Scholar
  22. 22.
    Hermann T, Bovermann T, Riedenklau E, Ritter H (2007) Tangible computing for interactive sonification of multivariate data. In: International workshop on interactive sonification, York, UK Google Scholar
  23. 23.
    Hermann T, Niehus C, Ritter H (2003) Interactive visualisation and sonification for monitoring complex processes. In: Proceedings of the 9th international conference on auditory display, Boston, USA Google Scholar
  24. 24.
    Hochenbaum J, Vallis O (2009) Bricktable: a musical tangible multi-touch interface. In: Proceedings of Berlin open conference 09, Berlin, Germany Google Scholar
  25. 25.
    Hochenbaum J, Vallis O, Diakopoulos D, Murphy J, Kapur A (2010) Designing expressive musical interfaces for tabletop surfaces. In: Proceedings of the 2010 international conference on new interfaces for musical expression, Sydney, Australia Google Scholar
  26. 26.
    Hunt A, Hermann T (2004) The importance of interaction in sonification. In: Proceedings of the 10th international conference on auditory display, Sydney, Australia Google Scholar
  27. 27.
    Judelman G (2004) Aesthetics and inspiration for visualisation design: Bridging the gap between art and science. In: International conference on information visualisation (IV’04), London, England Google Scholar
  28. 28.
    Kosara R (2007) Visualisation criticism—the missing link between information visualisation and art. In: International conference on information visualisation (IV’07), Zurich, Switzerland Google Scholar
  29. 29.
    Kramer G (1993) Auditory display: sonification, audification, and auditory interfaces, Perseus, New York Google Scholar
  30. 30.
    Kramer G, Walker BN, Bonebright T, Cook PR, Flowers JH, Miner N, Neuhoff JG (1997) Sonification report: status of the field and research agenda. Tech rep, National Science Foundation Google Scholar
  31. 31.
    Mayo E, Steinberg T (2010) The power of information: an independent review Google Scholar
  32. 32.
    Nees M, Walker B (2006) Relative intensity of auditory context for auditory graph design. In: 12th international conference on auditory display, London, UK, pp 95–98 Google Scholar
  33. 33.
    Pousman Z, Stasko J, Mateas M (2007) Casual information visualisation: depictions of data in everyday life. IEEE Trans Vis Comput Graph 13(6):1145–1152 CrossRefGoogle Scholar
  34. 34.
    Sack W (2006) Aesthetics of information visualisation. In: Paul C, Vesna V, Lovejoy M (eds) Context providers. MIT Press, Boston Google Scholar
  35. 35.
    Scha R, Bod R (1993) Computational esthetics. Inf Infbeleid 11(1):54–63 Google Scholar
  36. 36.
    Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualisations. In: IEEE symposium on visual languages, Boulder, CO, USA Google Scholar
  37. 37.
    Song H, Beilharz K (2007) Concurrent auditory stream discrimination in auditory graphing. J Comput 3(1):79–87 Google Scholar
  38. 38.
    Tufte ER (1992) Envisioning information. Graphics Press, Cheshire Google Scholar
  39. 39.
    Tünnermann R, Hermann T (2009) Multi-touch interactions for model-based sonification. In: Proceedings of the 15th international conference on auditory display, Copenhagen, Denmark Google Scholar
  40. 40.
    Moere Vande A (2005) Form follows data: the symbiosis between design and information visualisation. In: International conference on computer-aided architectural design (CAADfutures’05). OKK Verlag, Vienna Google Scholar
  41. 41.
    Vesna V (2007) Seeing the world in a grain of sand: the database aesthetics of everything. In: Vesna V (ed) Database aesthetics—art in the age of information overflow, electronic mediations. University of Minnesota Press, Minneapolis Google Scholar
  42. 42.
    Viegas F, Wattenberg M (2007) Artistic data visualisation: beyond visual analytic. In: Online communities and social computing. Lecture notes in computer science, vol 4564(15). Springer, Berlin, pp 182–191 CrossRefGoogle Scholar
  43. 43.
    Walker BN, Nees MA (2005) An agenda for research and development of multimodal graphs. In: Proceedings of the 11th international conference on auditory display, Limerick, Ireland Google Scholar
  44. 44.
    Wilkinson L (2005) The grammar of graphics, 2nd edn. Springer, Berlin zbMATHGoogle Scholar
  45. 45.
    Zacks J, Levy E, Tverksy B, Schiano D (1998) Reading bar graphs: effects of extraneous depth cues and graphical context. J Exp Psychol, Appl 4(2):119–138 CrossRefGoogle Scholar
  46. 46.
    Zacks J, Levy E, Tversky B, Schiano D (2001) Graphs in print. In: Olivier P, Anderson M, Meyer B (eds) Diagrammatic representation and reasoning. Springer, London Google Scholar

Copyright information

© OpenInterface Association 2011

Authors and Affiliations

  • Sam Ferguson
    • 1
  • Kirsty Beilharz
    • 1
  • Claudia A. Calò
    • 1
  1. 1.Faculty of Design, Architecture and BuildingUniversity of TechnologySydneyAustralia

Personalised recommendations