ImmertableApp: Interactive and Tangible Learning Music Environment

  • Sandra BaldassarriEmail author
  • Javier Marco
  • Clara Bonillo
  • Eva Cerezo
  • José Ramón Beltrán
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9733)


This paper presents ImmertableApp, an innovative multimodal interface based in tangible interaction in which audio edition is managed through physical controllers. The system is composed by two different main components: a tangible tabletop interface in which the sound parameters can be changed by the manipulation of physical controllers; and a graphic editor interface for setting the configuration of the controllers and their corresponding parameters through a tablet device. In this way ImmertableApp adds to a musical tangible interface the new possibilities of software interfaces: personalization for being adapted to different users, either experts or beginners and, so that it can be used as a didactic tool of different concepts related with sound synthesis. The system has been early assessed with experts in order to obtain feedback about its utility in different fields of music education. The results of the evaluation give the basis for interesting improvements in future versions.


Tangible user interface Music Audio edition Education Object manipulation Tablet 



Authors want to thank the ideas given by the ten music teachers that participated in the evaluation session and the support and collaboration of the Education Faculty professors of the University of Zaragoza: Marta Liesa, Sandra Vázquez and Ana Cristina Blasco. This work has been partly financed by the Spanish “Ministerio de Economía y Competitividad” through project No. TIN2015-67149-C3-1R and by the “Diputación General de Aragón” through project: ImmertableApp No. 1004460/2015.


  1. 1.
    Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Syst. J. 39(3.4), 915–931 (2000)CrossRefGoogle Scholar
  2. 2.
    Smithson Martin. Accessed 15 Feb 2016
  3. 3.
    Smartable: Gorenje design studio. Accessed 15 Feb 2016
  4. 4.
    Kaltenbranner, M., Jorda, S., Geiger, G., Alonso, M.: The reactable*: a collaborative musical instrument. In: 15th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, WETICE 2006, pp. 406–411 (2006)Google Scholar
  5. 5.
    Marco, J., Cerezo, E., Baldassarri, S., Mazzone, E., Read, J.C.: Bringing tabletop technologies to Kindergarten children. In: Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology, pp. 103–111. British Computer Society (2009)Google Scholar
  6. 6.
    Li, Y., Fontijn, W., Markopoulos, P.: A tangible tabletop game supporting therapy of children with cerebral palsy. In: Markopoulos, P., de Ruyter, B., IJsselsteijn, W.A., Rowland, D. (eds.) Fun and Games 2008. LNCS, vol. 5294, pp. 182–193. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Al Mahmud, A., Mubin, O., Shahid, S., Martens, J.B.: Designing and evaluating the tabletop game experience for senior citizens. In: Proceeding of the NordiCHI, 20–22 October 2008Google Scholar
  8. 8.
    Lyons, M.J., Mulder, A., Fels, S.: Introduction to designing and building musical interfaces. In: Proceedings of the Extended Abstracts of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM (2014)Google Scholar
  9. 9.
    Morreale, F., De Angeli, A., O’Modhrain, S.: Musical interface design: an experience-oriented framework. In: Proceedings of NIME, vol. 14, pp. 467–472 (2014)Google Scholar
  10. 10.
    Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reactable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, pp. 139–146. ACM (2007)Google Scholar
  11. 11.
    Schiettecatte, B., Vanderdonckt, J.: AudioCubes: a distributed cube tangible interface based on interaction range for sound design. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, pp. 3–10. ACM (2008)Google Scholar
  12. 12.
    Nielsen, J., Bærendsen, N.K., Jessen, C.: RoboMusicKids. In: IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning, DIGITEL 2008, pp. 149–156 (2008). doi: 10.1109/DIGITEL.2008.25
  13. 13.
    Potidis, S., Spyrou, T.: Spyractable: a tangible user interface modular synthesizer. In: Kurosu, M. (ed.) HCI 2014, Part II. LNCS, vol. 8511, pp. 600–611. Springer, Heidelberg (2014)Google Scholar
  14. 14.
    Parra-Damborenea, J.: Reactblocks: A 3D tangible interface for music learning. Master thesis, University of Pompeu Fabra (2014)Google Scholar
  15. 15.
    Costanza, E., Shelley, S.B., Robinson, J.: Introducing audio d-touch: a tangible user interface for music composition and performance. In: Proceedings in Human Computer Interaction (HCI). ACM (2003)Google Scholar
  16. 16.
    Bischof, M., Conradi, B., Lachenmaier, P., Linde, K., Meier, M., Pötzl, P., André, E.: Xenakis: combining tangible interaction with probability-based musical composition. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, pp. 121–124. ACM (2008)Google Scholar
  17. 17.
    Francesconi, J.I., Larrea, M., Manresa-Yee, C.: Tangible music composer for children. J. Comput. Sci. Tech. 13, 84–90 (2013)Google Scholar
  18. 18.
    Patten, J., Recht, B., Ishii, H.: Audiopad: a tag-based interface for musical performance. In: Proceedings of the 2002 Conference on New Interfaces for Musical Expression (2002)Google Scholar
  19. 19.
    Reactivision. Accessed 15 Feb 2016
  20. 20.
    Marco, J., Baldassarri, S., Cerezo, E.: ToyVision: a toolkit to support the creation of innovative board-games with tangible interaction. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, pp. 291–298. ACM (2013)Google Scholar
  21. 21.
    PureData. Accessed 15 Feb 2016
  22. 22.
    Roads, C.: The Computer Music Tutorial. MIT press, Cambridge (1996)Google Scholar
  23. 23.
    Schöning, J., Brandl, P., Daiber, F., Echtler, F., Hilliges, O., Hook, J., von Zadow, U.: Multi-touch surfaces: a technical guide. IEEE Tabletops Interact. Surf. 2, 11 (2008)Google Scholar
  24. 24.
    Brooke, J.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)Google Scholar
  25. 25.
    Villafuerte, L., Markova, M., Jorda, S.: Acquisition of social abilities through musical tangible user interface: children with autism spectrum condition and the reactable. In: CHI 2012 Extended Abstracts on Human Factors in Computing Systems, pp. 745–760. ACM 2012Google Scholar
  26. 26.
    Chen, W.: Multitouch tabletop technology for people with autism spectrum disorder: a review of the literature. Procedia Comput. Sci. 14(2012), 198–207 (2012)CrossRefGoogle Scholar
  27. 27.
    Nikolaidou, G.N.: ComPLuS model: a new insight in pupils’ collaborative talk, actions and balance during a computer-mediated music task. Comput. Educ. 58(2), 740–765 (2012)CrossRefGoogle Scholar
  28. 28.
    Marco, J., Cerezo, E., Baldassarri, S.: Lowering the threshold and raising the ceiling of tangible expressiveness in hybrid board-games. Multimedia Tools Appl. 75(1), 425–463 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Sandra Baldassarri
    • 1
    Email author
  • Javier Marco
    • 1
  • Clara Bonillo
    • 1
  • Eva Cerezo
    • 1
  • José Ramón Beltrán
    • 2
  1. 1.GIGA AffectiveLab, Aragon Institute of Engineering Research (I3A)Universidad de ZaragozaZaragozaSpain
  2. 2.Department of Electronic Engineering and Communications, Aragon Institute of Engineering Research (I3A)Universidad de ZaragozaZaragozaSpain

Personalised recommendations