Skip to main content
SpringerLink
Log in

Sound/tracks: artistic real-time sonification of train journeys

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

We present an application of sonification in an artistic context, namely to augment the visual impressions of train journeys. While sonification and auditory displays are typically used as means to present data and to inform the user, our project sound/tracks aims at enhancing the visual experience of looking out of a moving train’s window at the passing landscape by adding a sound dimension. This allows for reflecting upon the visual impressions and deepening the state of contemplation. To this end, sound/tracks translates the perceived movement of the scenery and other visual impressions, such as passing trains, into music. The continuously changing view outside the window is captured with a camera and transformed into instantaneously played music. The application can be run on mobile phones with a built-in camera and on laptops with a Web-cam. The paper proposes and discusses different sonification approaches and presents different application scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Sound/tracks sonification methods. http://www.cp.jku.at/projects/soundtracks/methods.html

  2. Bresin R (2005) What is the color of that music performance. In: Proceedings of the international computer music conference, Barcelona, Spain

    Google Scholar 

  3. Collier WG, Hubbard TL (2004) Musical scales and brightness evaluations: Effects of pitch, direction, and scale mode. Music Sci 8:151–173

    Google Scholar 

  4. Datteri DL, Howard JN (2004) The sound of color. In: Proceedings of the 8th international conference on music perception & cognition, Evanston, IL, USA

    Google Scholar 

  5. Rusconi E, Kwan B, Giordano BL, Umiltà C, Butterworth B (2006) Spatial representation of pitch height: the SMARC effect. Cognition 99(2):113–129

    Article  Google Scholar 

  6. Gondry M (2002) Music video for “The Chemical Brothers—Star Guitar”

  7. Gröhn L (2007) Sound of paintings. http://www.synestesia.fi

  8. Jensenius AR (2007) Action—sound: developing methods and tools to study music-related body movement. PhD thesis, University of Oslo, Norway

  9. Jo K, Nagano N (2008) Monalisa: See the sound, hear the image. In: Proceedings of the 8th international conference on new interfaces for musical expression

    Google Scholar 

  10. Ängeslevä J, Cooper R (2005) Last clock. IEEE Comput Graph Appl 25(1):20–23

    Article  Google Scholar 

  11. Knees P, Pohle T, Widmer G Sound/tracks Web site. http://www.cp.jku.at/projects/soundtracks

  12. Krumhansl CL, Kessler EJ (1982) Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. Psychol Rev 89:334–368

    Article  Google Scholar 

  13. Marks LE (1974) On associations of light and sound: The mediation of brightness, pitch, and loudness. Am J Psychol 87(1–2):173–188

    Article  Google Scholar 

  14. Meijer PB The vOICe. http://www.seeingwithsound.com

  15. Meijer PB (1992) An experimental system for auditory image representations. IEEE Trans Biomed Eng 39(2):112–121

    Article  Google Scholar 

  16. Nunes M, Greenberg S, Carpendale S, Gutwin C (2007) What did I miss? Visualizing the past through video traces. In: Proceedings of the 10th European conference on computer supported cooperative work, Limerick, Ireland

    Google Scholar 

  17. Pohle T, Knees P, Widmer G (2008) Sound/tracks: real-time synaesthetic sonification and visualisation of passing landscapes. In: Proceedings of the 16th ACM conference on multimedia, Vancouver, BC, Canada

    Google Scholar 

  18. Reinhart M, Widrich V tx-transform. http://www.tx-transform.com

  19. Tanaka A (2004) Bondage. http://www.xmira.com/atau/bondage/

  20. Wikipedia (2008) Alexander Scriabin—Wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title=Alexander_Scriabin&oldid=189505713

Download references

Acknowledgements

This research is supported by the Austrian Science Fund (FWF) under project numbers L511-N15, TRP 109-N23, and Z159 (Wittgenstein Award).

Author information

Authors and Affiliations

  1. Department of Computational Perception, Johannes Kepler University, Altenberger Str. 69, 4040, Linz, Austria

    Peter Knees, Tim Pohle & Gerhard Widmer

Authors
  1. Peter Knees
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Pohle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard Widmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Knees.

Additional information

This article is a revised and updated version of the paper “sound/tracks: Real-Time Synaesthetic Sonification and Visualisation of Passing Landscapes” by the same authors originally published in the proceedings of the ACM Multimedia 2008 conference, cf. [17].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Knees, P., Pohle, T. & Widmer, G. Sound/tracks: artistic real-time sonification of train journeys. J Multimodal User Interfaces 6, 87–93 (2012). https://doi.org/10.1007/s12193-011-0089-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-011-0089-x

Keywords

Search

Navigation