Surface Interactions for Interactive Sonification

  • René Tünnermann
  • Lukas Kolbe
  • Till Bovermann
  • Thomas Hermann
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5954)


This paper presents novel interaction modes for Model-Based Sonification (MBS) via interactive surfaces. We first discuss possible interactions for MBS on a multi-touch surface. This is followed by a description of the Data Sonogram Sonification and the Growing Neural Gas Sonification Model and their implementation for the multi-touch interface. Modifications from the original sonification models such as the limited space scans are described and discussed with sonification examples. Videos showing interaction examples are provided. Furthermore, the presented system provides a basis for the implementation of known and novel sonification models. We discuss the available interaction modes with multi-touch surfaces and how these interactions can be profitably used to control spatial and non-spatial sonification models.


Sonification Model-Based Sonification Data Mining Interactive Surfaces 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bovermann, T., Hermann, T., Ritter, H.: A tangible environment for ambient data representation. In: First International Workshop on Haptic and Audio Interaction Design, August 2006, vol. 2, pp. 26–30 (2006)Google Scholar
  2. 2.
    Buxton, B.: Multi-touch systems that i have known and loved (2009),
  3. 3.
    Dourish, P.: Where the action is: The foundation of embodied interaction. MIT Press, Cambridge (2001)Google Scholar
  4. 4.
    Forina, M.: Arvus - an extendible package for data exploration, Classification and Correlation,
  5. 5.
    Han, J.Y.: Low-cost multi-touch sensing through frustrated total internal reflection. In: UIST 2005: Proceedings of the 18th annual ACM symposium on User interface software and technology, pp. 115–118. ACM Press, New York (2005)CrossRefGoogle Scholar
  6. 6.
    Fisher, R.A.: UCI Repository of Maschine Learning Databases – Iris Data Set (1999)Google Scholar
  7. 7.
    Fry, B., Reas, C.: Processing programminig environment (2001),
  8. 8.
    Heidegger, M.: Sein und Zeit. Niemeyer, Halle a. d. S. (1927)Google Scholar
  9. 9.
    Hermann, T.: Taxonomy and definitions for sonification and auditory display. In: Katz, B. (ed.) Proc. Int. Conf. Auditory Display (ICAD 2008), France (2008)Google Scholar
  10. 10.
    Hermann, T., Ritter, H.: Listen to your data: Model-based sonification for data analysis. In: Lasker, G.E. (ed.) Advances in intelligent computing and multimedia systems. Int. Inst. for Advanced Studies in System research and cybernetics, Baden-Baden, Germany, pp. 189–194 (1999)Google Scholar
  11. 11.
    Hermann, T.: Sonification for exploratory data analysis. PhD thesis, Bielefeld University (February 2002),
  12. 12.
    Hermann, T., Krause, J., Ritter, H.: Real-time control of sonification models with an audio-haptic interface. In: Nakatsu, R., Kawahara, H. (eds.) Proc. Int. Conf. Auditory Display (ICAD 2002), Kyoto, Japan, pp. 82–86 (2002)Google Scholar
  13. 13.
    Hermann, T., Milczynski, M., Ritter, H.: A malleable device with applications to sonification-based data exploration. In: Stockman, T. (ed.) Proc. Int. Conf. Auditory Display (ICAD 2006), pp. 69–76. University of London, London, UK (2006)Google Scholar
  14. 14.
    Hunt, A., Hermann, T., Pauletto, S.: Interacting with sonification systems: Closing the loop. In: Banissi, E., Börner, K. (eds.) IV 2004: Proceedings of the Information Visualisation, Eighth International Conference on (IV 2004), Washington, DC, USA, pp. 879–884. IEEE Computer Society, Los Alamitos (2004)CrossRefGoogle Scholar
  15. 15.
    Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics, 164–168 (1944)Google Scholar
  16. 16.
    Marquardt, D.: An algorithm for least-qquares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics 11, 431–441 (1963)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Kramer, G. (ed.): Auditory display: sonification, audification, and auditory interfaces. Addison-Wesley, Reading (1994)Google Scholar
  18. 18.
    Kramer, G., Walker, B., Bonebright, T., Cook, P., Flower, J., Miner, N., Neuhoff, J., Bargar, R., Barrass, S., Berger, J., Evreinov, G., Fitch, W.T., Grohn, M., Handel, S., Kaper, H., Levkowitz, H., Lodha, S., Shinn-Cunningham, B., Simoni, M., Tipei, S.: Sonification report: Status of the field and research agenda (1997)Google Scholar
  19. 19.
    McCartney, J.: SuperCollider hub. (July 2004)
  20. 20.
    Hermann, T., Ritter, H.: Neural gas sonification: Growing adaptive interfaces for interacting with data. In: Banissi, E., Börner, K. (eds.) IV 2004: Proceedings of the Information Visualisation, Eighth International Conference on (IV 2004), Washington, DC, USA, pp. 871–878. IEEE Computer Society, Los Alamitos (2004)CrossRefGoogle Scholar
  21. 21.
    Fritzke, B.: A growing neural gas network learns topologies. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 625–632. MIT Press, Cambridge (1995)Google Scholar
  22. 22.
    Kaltenbrunner, M., Bovermann, T., Bencina, R., Costanza, E.: TUIO: A protocol for table based tangible user interfaces. In: GW (2005)Google Scholar
  23. 23.
    Hansen, T., Hourcade, J.P., Virbel, M., Patali, S., Serra, T.: PyMT: A post-WIMP multi-touch user interface toolkit. In: Proceedings of the International Conference on Interactive Tabletops and Surfaces (ITS 2009), Banff, Canada (2009)Google Scholar
  24. 24.
    Zito, T., Wilbert, N., Wiskott, L., Berkes, P.: Modular toolkit for data processing (MDP): A python data processing frame work. Frontiers in Neuroinformics (2008),
  25. 25.
    Stinson, P.K.: SCOSC: SuperCollider OSC interface for python (October 2009),

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • René Tünnermann
    • 1
  • Lukas Kolbe
    • 1
  • Till Bovermann
    • 1
  • Thomas Hermann
    • 1
  1. 1.Ambient Intelligence GroupCognitive Interaction Technology - Center of Excellence (CITEC)BielefeldGermany

Personalised recommendations