Sound/tracks: artistic real-time sonification of train journeys

  • Peter KneesEmail author
  • Tim Pohle
  • Gerhard Widmer
Original Paper


We present an application of sonification in an artistic context, namely to augment the visual impressions of train journeys. While sonification and auditory displays are typically used as means to present data and to inform the user, our project sound/tracks aims at enhancing the visual experience of looking out of a moving train’s window at the passing landscape by adding a sound dimension. This allows for reflecting upon the visual impressions and deepening the state of contemplation. To this end, sound/tracks translates the perceived movement of the scenery and other visual impressions, such as passing trains, into music. The continuously changing view outside the window is captured with a camera and transformed into instantaneously played music. The application can be run on mobile phones with a built-in camera and on laptops with a Web-cam. The paper proposes and discusses different sonification approaches and presents different application scenarios.


Mobile music generation Sonification Train journey 



This research is supported by the Austrian Science Fund (FWF) under project numbers L511-N15, TRP 109-N23, and Z159 (Wittgenstein Award).


  1. 1.
    Sound/tracks sonification methods.
  2. 2.
    Bresin R (2005) What is the color of that music performance. In: Proceedings of the international computer music conference, Barcelona, Spain Google Scholar
  3. 3.
    Collier WG, Hubbard TL (2004) Musical scales and brightness evaluations: Effects of pitch, direction, and scale mode. Music Sci 8:151–173 Google Scholar
  4. 4.
    Datteri DL, Howard JN (2004) The sound of color. In: Proceedings of the 8th international conference on music perception & cognition, Evanston, IL, USA Google Scholar
  5. 5.
    Rusconi E, Kwan B, Giordano BL, Umiltà C, Butterworth B (2006) Spatial representation of pitch height: the SMARC effect. Cognition 99(2):113–129 CrossRefGoogle Scholar
  6. 6.
    Gondry M (2002) Music video for “The Chemical Brothers—Star Guitar” Google Scholar
  7. 7.
    Gröhn L (2007) Sound of paintings.
  8. 8.
    Jensenius AR (2007) Action—sound: developing methods and tools to study music-related body movement. PhD thesis, University of Oslo, Norway Google Scholar
  9. 9.
    Jo K, Nagano N (2008) Monalisa: See the sound, hear the image. In: Proceedings of the 8th international conference on new interfaces for musical expression Google Scholar
  10. 10.
    Ängeslevä J, Cooper R (2005) Last clock. IEEE Comput Graph Appl 25(1):20–23 CrossRefGoogle Scholar
  11. 11.
    Knees P, Pohle T, Widmer G Sound/tracks Web site.
  12. 12.
    Krumhansl CL, Kessler EJ (1982) Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. Psychol Rev 89:334–368 CrossRefGoogle Scholar
  13. 13.
    Marks LE (1974) On associations of light and sound: The mediation of brightness, pitch, and loudness. Am J Psychol 87(1–2):173–188 CrossRefGoogle Scholar
  14. 14.
    Meijer PB The vOICe.
  15. 15.
    Meijer PB (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39(2):112–121 CrossRefGoogle Scholar
  16. 16.
    Nunes M, Greenberg S, Carpendale S, Gutwin C (2007) What did I miss? Visualizing the past through video traces. In: Proceedings of the 10th European conference on computer supported cooperative work, Limerick, Ireland Google Scholar
  17. 17.
    Pohle T, Knees P, Widmer G (2008) Sound/tracks: real-time synaesthetic sonification and visualisation of passing landscapes. In: Proceedings of the 16th ACM conference on multimedia, Vancouver, BC, Canada Google Scholar
  18. 18.
    Reinhart M, Widrich V tx-transform.
  19. 19.
    Tanaka A (2004) Bondage.
  20. 20.
    Wikipedia (2008) Alexander Scriabin—Wikipedia, the free encyclopedia.

Copyright information

© OpenInterface Association 2012

Authors and Affiliations

  1. 1.Department of Computational PerceptionJohannes Kepler UniversityLinzAustria

Personalised recommendations