Interactive Sonification Markup Language (ISML) for Efficient Motion-Sound Mappings

  • James Walker
  • Michael T. Smith
  • Myounghoon JeonEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9170)


Despite rapid growth of research on auditory display and sonification mapping per se, there has been little effort on efficiency or accessibility of the mapping process. In order to expedite variations on sonification research configurations, we have developed the Interactive Sonification Markup Language (ISML). ISML is designed within the context of the Immersive Interactive Sonification Platform (iISoP) at Michigan Technological University. We present an overview of the system, the motivation for developing ISML, and the time savings realized through its development. We then discuss the features of ISML and its accompanying graphical editor, and conclude by summarizing the system’s feature development and future plans for its further enhancement. ISML is expected to decrease repetitive development tasks for multiple research studies and to increase accessibility to diverse sonification researchers who do not have programming experience.


Design research Interactive sonification Sonification markup language 


  1. 1.
    Hermann, T., Hunt, A.: The discipline of interactive sonification. In: Proceedings of the International Workshop on Interactive Sonification (ISon), Bielefeld, Germany (2004)Google Scholar
  2. 2.
    Kramer, G., et al.: The sonification report: status of the field and research agenda. Report prepared for the National Science Foundation by Members of the International Community for Auditory Display (ICAD), Santa Fe, NM, USA (1999)Google Scholar
  3. 3.
    Raman, P., Davison, B.K., Jeon, M., Walker, B.N.: Reducing repetitive development tasks in auditory menu displays with the auditory menu library. In: Proceedings of the International Conference on Auditory Display (ICAD), pp. 229–237, Washington, D.C., USA (2010)Google Scholar
  4. 4.
    Antle, A.N., Droumeva, M., Corness, G.: Playing with the sound maker: do embodied metaphors help children learn? In: Proceedings of the International Conference on Interaction Design and Children, pp. 178–185, Chicago, IL, USA (2008)Google Scholar
  5. 5.
    Ferguson, S.: Learning musical instrument skills through interactive sonification. In: Proceedings of the International Conference on New Interfaces for Musical Expression (NIME2015), pp. 384–389 (2006)Google Scholar
  6. 6.
    Howison, M., Trninic, D., Reinholz, D., Abrahamson, D.: The mathematical imagery trainer: from embodied interaction to conceptual learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI2011), pp. 1989–1998, Vancouver, BC, Canada (2011)Google Scholar
  7. 7.
    Walker, B.N., Godfrey, M.T., Orlosky, J.E., Bruce, C.M., Sanford, J.: Aquarium sonification: Soundscapes for accessible dynamic informal learning environments. In: Proceedings of the International Conference on Auditory Display (ICAD 2006), pp. 238–241, London, UK (2006)Google Scholar
  8. 8.
    Jeon, M., Winton, R.J., Henry, A.G., Oh, S., Bruce, C.M., Walker, B.N.: Designing interactive sonification for live aquarium exhibits. In: Stephanidis, C. (ed.) HCII 2013, Part I. CCIS, vol. 373, pp. 332–336. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  9. 9.
    Jeon, M., Winton, R.J., Yim, J.B., Bruce, C.M., Walker, B.N.: Aquarium fugue: interactive sonification for children and visually impaired audience in informal learning environments. In: Proceedings of the 18th International Conference on Auditory Display (ICAD 2012), Atlanta, GA, USA (2012)Google Scholar
  10. 10.
    Jeon, M., Smith, M.T., Walker, J.W., Kuhl, S.A.: Constructing the immersive interactive sonification platform (iISoP). In: Streitz, N., Markopoulos, P. (eds.) DAPI 2014. LNCS, vol. 8530, pp. 337–348. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  11. 11.
    Hermann, T.: Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 14th International Conference on Auditory Display (ICAD 2008), Paris, France (2008)Google Scholar
  12. 12.
    Coombs, J.H., Renear, A.H., DeRose, S.J.: Markup systems and the future of scholarly text processing. Commun. ACM 30(11), 933–947 (1987)CrossRefGoogle Scholar
  13. 13.
    World Wide Web Consortium: Extensible Markup Language (XML) 1.0 (5th edn.) 2008. Web. 4 Oct 2014Google Scholar
  14. 14.
    Clark, J.: Comparison of SGML and XML. World Wide Web Consortium, 1997. Web. 4 Oct 2014Google Scholar
  15. 15.
    Philip, F.: Extremes of XML. XML London, 15–16 June 2013Google Scholar
  16. 16.
    Wang, G.: The ChucK Audio Programming Language. Dissertation, Princeton University, Princeton (2008)Google Scholar
  17. 17.
    Boulanger, R.: The Csound Book. MIT Press, Cambridge (2000)Google Scholar
  18. 18.
    Scheirer, E.D.: The MPEG-4 structured audio standard. In: Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal processing 1998, vol. 6. IEEE (1998)Google Scholar
  19. 19.
    Faste, T., Faste, H.: Demystifying ‘design research’: design is not research, research is design. IDSA (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • James Walker
    • 1
  • Michael T. Smith
    • 1
  • Myounghoon Jeon
    • 1
    Email author
  1. 1.Mind Music Machine LabMichigan Technological UniversityHoughtonUSA

Personalised recommendations