Skip to main content
SpringerLink
Log in

Design, Implementation, and Early Experimentation of a Music Tangible User Interface for Elderly People Rehabilitation

  • Conference paper
  • First Online:
Computer-Human Interaction Research and Applications (CHIRA 2021, CHIRA 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1882))

  • 192 Accesses

Abstract

This article proposes an innovative approach for cognitive and motor rehabilitation of elderly individuals that combines tangible user interfaces, digital technologies, and musical expression. After reviewing the current literature on age-related impairments and the use of tangible user interfaces in rehabilitation, we introduce a specific MIDI controller called Kibo that employs geometric fiducials called tangibles. Thanks to Bluetooth connectivity and the adoption of the Web MIDI API, this device communicates with a specially-designed web framework that includes three games to facilitate the development or recovery of cognitive and motor abilities. The feedback obtained in an early experimentation phase from domain experts and a focus group underlined some strengths and weaknesses, thus driving the first revision of the interface and the gameplay. This work will describe the design and implementation of the early prototype and will shed light on the future evolution of the web platform. From the tests conducted so far, this solution based on a musical controller equipped with tangibles exhibits the potential to enhance the effectiveness and enjoyment of rehabilitation programs for elderly individuals.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.who.int/news-room/fact-sheets/detail/ageing-and-health.

  2. 2.

    https://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopulationAgeing2019-Highlights.pdf.

  3. 3.

    https://youtu.be/sKojCxcpgnk.

  4. 4.

    https://youtu.be/ZQdIxQ_EOLI.

  5. 5.

    https://www.kodaly.app/.

  6. 6.

    World Wide Web Consortium, https://www.w3.org/.

  7. 7.

    https://www.avid.com/pro-tools.

References

  1. Adlakha, S., Chhabra, D., Shukla, P.: Effectiveness of gamification for the rehabilitation of neurodegenerative disorders. Chaos, Solitons Fractals 140, 110192 (2020)

    Article  Google Scholar 

  2. American Music Therapy Association: About music therapy & AMTA. https://www.musictherapy.org/about/ (2022) Accessed May 27 2021

  3. Amico, M.D., Ludovico, L.A.: Kibo: A MIDI controller with a tangible user interface for music education. In: International Conference on Computer Supported Education, pp. 613–619. SCITEPRESS (2020)

    Google Scholar 

  4. Baratè, A., Korsten, H., Ludovico, L.A.: A music tangible user interface for the cognitive and motor rehabilitation of elderly people. In: Constantine, L., Holzinger, A., Silva, H.P., Vanderdonckt, J. (eds.) Proceedings of the 6th International Conference on Computer-Human Interaction Research and Applications (CHIRA 2022), pp. 121–128. SCITEPRESS - Science and Technology Publications, Lda. (2022). https://doi.org/10.5220/0011395900003323

  5. Baratè, A., Ludovico, L.A.: Web MIDI API: State of the art and future perspectives. J. Audio Eng. Society 70(11), 918–925 (2022). https://doi.org/10.17743/jaes.2022.0028

  6. Baratè, A., Ludovico, L.A., Malchiodi, D.: Fostering computational thinking in primary school through a LEGO®-based music notation. Procedia Comput. Sci. Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 21st International Conference, KES 2017, 6–8 September 2017, Marseille, France 112, 1334–1344 (2017). https://doi.org/10.1016/j.procs.2017.08.018

  7. Baratè, A., Ludovico, L.A., Oriolo, E.: An ensemble of tangible user interfaces to foster music awareness and interaction in vulnerable learners. In: Proceedings of 5th International Conference on Computer-Human Interaction Research and Applications, pp. 48–57. SCITEPRESS (2021)

    Google Scholar 

  8. Bartolomeu, P., Fonseca, J., Duarte, P., Rodrigues, P., Girao, L.: Midi over bluetooth. In: 2005 IEEE Conference on Emerging Technologies and Factory Automation. vol. 1, pp. 8–102 (2005). https://doi.org/10.1109/ETFA.2005.1612507

  9. Berton, A., et al.: Virtual reality, augmented reality, gamification, and telerehabilitation: psychological impact on orthopedic patients’ rehabilitation. J. Clin. Med. 9(8), 2567 (2020)

    Article  Google Scholar 

  10. Burke, J., McNeill, M., Charles, D., Morrow, P., Crosbie, J., McDonough, S.: Augmented reality games for upper-limb stroke rehabilitation. In: 2010 Second International Conference on Games and Virtual Worlds for Serious Applications, pp. 75–78 (2010). https://doi.org/10.1109/VS-GAMES.2010.21

  11. de Castro-Cros, M., et al.: Effects of gamification in BCI functional rehabilitation. Front. Neurosci. 14, 882 (2020)

    Article  Google Scholar 

  12. Cerezo, E., Bonillo, C., Baldassarri, S.: Therapeutic activities for elderly people based on tangible interaction. In: ICT4AWE, pp. 281–290 (2020)

    Google Scholar 

  13. Chu, R.J.c.: How family support and internet self-efficacy influence the effects of e-learning among higher aged adults-analyses of gender and age differences. Comput. Educ. 55(1), 255–264 (2010)

    Google Scholar 

  14. Czaja, S.J., Lee, C.C.: The impact of aging on access to technology. Univ. Access Inf. Soc. 5(4), 341–349 (2007)

    Article  Google Scholar 

  15. Czaja, S.J., Sharit, J.: Age differences in attitudes toward computers. J. Gerontol. B Psychol. Sci. Soc. Sci. 53(5), P329–P340 (1998)

    Article  Google Scholar 

  16. De Carvalho, R.N.S., Ishitani, L., Nogueira Sales De Carvalho, R., et al.: Motivational factors for mobile serious games for elderly users. In: Proceedings of XI SB games, pp. 2–4 (2012)

    Google Scholar 

  17. Dörner, R., Göbel, S., Effelsberg, W., Wiemeyer, J. (eds.): Serious Games. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40612-1

    Book  Google Scholar 

  18. Duckworth, J., et al.: Resonance: an interactive tabletop artwork for co-located group rehabilitation and play. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2015. LNCS, vol. 9177, pp. 420–431. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20684-4_41

    Chapter  Google Scholar 

  19. Farage, M.A., Miller, K.W., Ajayi, F., Hutchins, D.: Design principles to accommodate older adults. Global J. Health Sci. 4(2), 2 (2012)

    Article  Google Scholar 

  20. Fonseca, X., Slingerland, G., Lukosch, S., Brazier, F.: Designing for meaningful social interaction in digital serious games. Entertain. Comput. 36, 100385 (2021)

    Article  Google Scholar 

  21. Gohlke, K., Hlatky, M., de Jong, B.: Physical construction toys for rapid sketching of tangible user interfaces. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 643–648. TEI ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2677199.2687900

  22. Huppert, E., et al.: The development of children’s preferences for equality and equity across 13 individualistic and collectivist cultures. Dev. Sci. 22(2), e12729 (2019)

    Article  Google Scholar 

  23. Iachini, T., Borghi, A.M., Senese, V.P.: Categorization and sensorimotor interaction with objects. Brain Cogn. 67(1), 31–43 (2008)

    Article  Google Scholar 

  24. Ijsselsteijn, W., Nap, H.H., de Kort, Y., Poels, K.: Digital game design for elderly users. In: Proceedings of the 2007 Conference on Future Play, pp. 17–22. Future Play ’07, Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1328202.1328206

  25. Ishii, H.: Tangible bits: Beyond pixels. In: Proceedings of 2nd International Conference on Tangible and Embedded Interaction, pp. xv–xxv. TEI ’08, Association for Computing Machinery (2008). https://doi.org/10.1145/1347390.1347392

  26. Ishii, H.: The tangible user interface and its evolution. Commun. ACM 51(6), 32–36 (2008). https://doi.org/10.1145/1349026.1349034

  27. Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 234–241 (1997)

    Google Scholar 

  28. Iversen, T.R.: Exploring tangible interaction: Alternative interfaces for assisting elderly users. Master’s thesis, Department of Informatics, University of Oslo (2015)

    Google Scholar 

  29. Jankovic, J.: Parkinson’s disease: clinical features and diagnosis. J. Neurol., Neurosurgery Psych. 79(4), 368–376 (2008)

    Article  Google Scholar 

  30. Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of 1st International Conference on Tangible and Embedded Interaction, pp. 139–146 (2007)

    Google Scholar 

  31. Laguna, K., Babcock, R.L.: Computer anxiety in young and older adults: implications for human-computer interactions in older populations. Comput. Hum. Behav. 13(3), 317–326 (1997)

    Article  Google Scholar 

  32. Maylor, E.A.: Prospective memory in normal ageing and dementia. Neurocase 1(3), 285–289 (1995)

    Article  Google Scholar 

  33. Miotti, B., Bassani, L., Cauteruccio, E., Morandi, M.: Musicblocks: An innovative tool for learning the foundations of music. In: Proceedings of CSME 2022, pp. 475–484 (2022). https://doi.org/10.5220/0011152100003182

  34. Newton-Dunn, H., Nakano, H., Gibson, J.: Block jam: a tangible interface for interactive music. J. New Music Res. 32(4), 383–393 (2003)

    Article  Google Scholar 

  35. Paradiso, J.A., Hsiao, K.y., Benbasat, A.: Tangible music interfaces using passive magnetic tags. In: Proceedings of the 2001 Conference on New Interfaces for Musical Expression, pp. 1–4. NIME ’01, National University of Singapore, SGP (2001)

    Google Scholar 

  36. Pearce, C.: The truth about baby boomer gamers: A study of over-forty computer game players. Games and Culture 3(2), 142–174 (2008). https://doi.org/10.1177/1555412008314132, https://doi.org/10.1177/1555412008314132

  37. Rego, P.A., Moreira, P.M., Reis, L.P.: Architecture for serious games in health rehabilitation. In: Rocha, Á., Correia, A.M., Tan, F.B., Stroetmann, K.A. (eds.) New Perspectives in Information Systems and Technologies, Volume 2. AISC, vol. 276, pp. 307–317. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-05948-8_30

    Chapter  Google Scholar 

  38. Salthouse, T.A.: Mediation of adult age differences in cognition by reductions in working memory and speed of processing. Psychol. Sci. 2(3), 179–183 (1991)

    Article  Google Scholar 

  39. Schiettecatte, B., Vanderdonckt, J.: Audiocubes: A distributed cube tangible interface based on interaction range for sound design. In: Proceedings of 2nd International Conference on Tangible and Embedded Interaction, pp. 3–10. TEI ’08, Association for Computing Machinery, New York, NY, USA (2008). https://doi.org/10.1145/1347390.1347394

  40. Scott, A.J.: The longevity society. Lancet Healthy Longevity 2(12), e820–e827 (2021)

    Article  Google Scholar 

  41. Steiner, B., Elgert, L., Saalfeld, B., Wolf, K.H., et al.: Gamification in rehabilitation of patients with musculoskeletal diseases of the shoulder: scoping review. JMIR Serious Games 8(3), e19914 (2020)

    Article  Google Scholar 

  42. Tuah, N.M., Ahmedy, F., Gani, A., Yong, L.N.: A survey on gamification for health rehabilitation care: applications, opportunities, and open challenges. Information 12(2), 91 (2021)

    Article  Google Scholar 

  43. Vandermaesen, M., De Weyer, T., Feys, P., Luyten, K., Coninx, K.: Integrating serious games and tangible objects for functional handgrip training: A user study of handly in persons with multiple sclerosis. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, pp. 924–935 (2016)

    Google Scholar 

  44. Wecker, N.S., Kramer, J.H., Hallam, B.J., Delis, D.C.: Mental flexibility: age effects on switching. Neuropsychology 19(3), 345 (2005)

    Article  Google Scholar 

  45. World Health Organization, United Nations Children’s Fund: Global report on assistive technology. World Health Organization (2022), https://apps.who.int/iris/rest/bitstreams/1424204/retrieve

  46. Zhao, X., et al.: Segara: Integrating serious games and handgrip for hand rehabilitation in rheumatoid arthritis patients. In: The 9th International Symposium of Chinese CHI, pp. 101–104 (2021)

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to Kodaly S.r.l., the manufacturer of Kibo, for the support. The authors also wish to thank the experts of Fondazione Don Carlo Gnocchi, Milan for their help in the early evaluation of the prototype and all elderly people participating in the focus groups.

Author information

Authors and Affiliations

  1. Laboratory of Music Informatics, Department of Computer Science, University of Milan, via G. Celoria 18, Milan, Italy

    Helene Korsten, Adriano Baratè & Luca A. Ludovico

Authors
  1. Helene Korsten
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriano Baratè
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca A. Ludovico
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca A. Ludovico .

Editor information

Editors and Affiliations

  1. Medical University Graz, Graz, Austria

    Andreas Holzinger

  2. IT- Institute of Telecommunications, Lisbon, Portugal

    Hugo Plácido da Silva

  3. Université Catholique de Louvain, Louvain-la-Neuve, Belgium

    Jean Vanderdonckt

  4. Madeira Interactive Technologies Institute, Funchal, Portugal

    Larry Constantine

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Korsten, H., Baratè, A., Ludovico, L.A. (2023). Design, Implementation, and Early Experimentation of a Music Tangible User Interface for Elderly People Rehabilitation. In: Holzinger, A., da Silva, H.P., Vanderdonckt, J., Constantine, L. (eds) Computer-Human Interaction Research and Applications. CHIRA CHIRA 2021 2022. Communications in Computer and Information Science, vol 1882. Springer, Cham. https://doi.org/10.1007/978-3-031-41962-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-41962-1_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-41961-4

  • Online ISBN: 978-3-031-41962-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation