Skip to main content
SpringerLink
Log in

Sonification and interaction design in computer games for visually impaired individuals

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video games are changing how we interact and communicate with each other. They can provide an authentic and collaborative platform for building new communities and connecting people. Since video games generally rely on visual elements that must be recognized by the players, most of them are not accessible by visually impaired people. In this research, we study the sonification and interaction issues in the design of computer games for visually impaired individuals. We have proposed an audio game called GrandEscape with the focus on the special needs of visually impaired people while playing. A comprehensive set of user studies has been performed to evaluate different interaction and sonification techniques in terms of providing a sense of presence and gaming experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ankolekar A, Sandholm T, Yu L (2013) Play it by ear: a case for serendipitous discovery of places with musicons. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp 2959–2968

  2. Atkinson MT, Gucukoglu S, Machin CH, Lawrence AE (2006) Making the mainstream accessible: redefining the game. In: Proceedings of the 2006 ACM SIGGRAPH symposium on Videogames, pp 21–28

  3. Balan O, Moldoveanu A, Moldoveanu F (2015) Navigational audio games: an effective approach toward improving spatial contextual learning for blind people. Int J Disability Human Dev 14(2):109–118

    Article  Google Scholar 

  4. Berge D, Bettencourt D, Lageweg S, Overman W, Zaidi A, Bidarra R (2020) Pinball for the Visually Impaired–an Audio Spatialization and Sonification Mobile Game. In: Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play, pp 43–46

  5. Bhagat S, Jeong EJ, Kim DJ (2020) The role of individuals’ need for online social interactions and interpersonal incompetence in digital game addiction. Int J Human–Comput Interaction 36(5):449–463

    Article  Google Scholar 

  6. Borges OT, Oliveira JD, de Borba Campos M, Marczak S (2018) Fair Play: A Guidelines Proposal for the Development of Accessible Audiogames for Visually Impaired Users. In: International conference on universal access in human-computer interaction. Springer, Cham, pp 401–419

  7. Borges OT, de Borba Campos M (2017) “I’m blind, can I play?” Recommendations for the development of audiogames. In: International Conference on Universal Access in Human-Computer Interaction. Springer, Cham, pp 351–365

  8. Brame CM, Martin D, Martin P (1998) Counseling the blind or visually impaired child: An examination of behavioral techniques. Prof School Couns 1 (5):60–62

    Google Scholar 

  9. Brewster SA (1998) Using nonspeech sounds to provide navigation cues. ACM Trans Comput-Human Interaction (TOCHI) 5(3):224–259

    Article  Google Scholar 

  10. Buaud A, Svensson H, Archambault D, Burger D (2002) Multimedia games for visually impaired children. In: International conference on computers for handicapped persons. Springer, Berlin, pp 173–180

  11. Caltenco H, Magnusson C, Rydeman B, Finocchietti S, Cappagli G, Cocchi E, Gori M (2016) Co-designing wearable technology together with visually impaired children. Int J Mob Human Comput Interaction (IJMHCI) 8 (4):68–84

    Article  Google Scholar 

  12. Chen J (2007) Flow in games (and everything else). Commun ACM 50(4):31–34

    Article  Google Scholar 

  13. Chen L, Hussain I, Chen R, Huang W, Chen G (2013) BlueView: a perception assistant system for the visually impaired. In: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, pp. 143–146

  14. Chen HE, Lin YY, Chen CH, Wang IF (2015) BlindNavi: A navigation app for the visually impaired smartphone user. In: Proceedings of the 33rd annual ACM conference extended abstracts on human factors in computing systems, pp 19–24

  15. Chion M (2019) Audio-vision: sound on screen. Columbia University Press

  16. de Borba Campos M, Oliveira JD (2016) Usability, accessibility and gameplay heuristics to evaluate audiogames for users who are blind. In: International conference on universal access in human-computer interaction. Springer, Cham, pp 38–48

  17. Drossos K, Zormpas N, Giannakopoulos G, Floros A (2015) Accessible games for blind children, empowered by binaural sound. In: Proceedings of the 8th ACM International conference on PErvasive technologies related to assistive environments, pp 1–8

  18. Dwyer P, VanLund P (2008) The last crusade

  19. Engström H, Brusk J, Östblad PA (2015) Including visually impaired players in a graphical adventure game: A study of immersion. IADIS Int J Comput Sci Inf Syst 10(2):95–112

    Google Scholar 

  20. Folmer E (2018) -Video Games for Users with Visual Impairments. In: Assistive Technology for Blindness and Low Vision. . CRC Press, pp 359–382

  21. Freeman E, Wilson G, Brewster S, Baud-Bovy G, Magnusson C, Caltenco H (2017) Audible beacons and wearables in schools: helping young visually impaired children play and move independently. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp 4146–4157

  22. Friberg J, Gärdenfors D (2004) Audio games: new perspectives on game audio. In: Proceedings of the 2004 ACM SIGCHI International conference on advances in computer entertainment technology, pp 148–154

  23. Friberg J, Gärdenfors D (2004) Audio games: new perspectives on game audio. In: Proceedings of the 2004 ACM SIGCHI International conference on advances in computer entertainment technology, pp 148–154

  24. Gaudy T, Natkin S, Archambault D (2009) Pyvox 2: an audio game accessible to visually impaired people playable without visual nor verbal instructions. In: Transactions on edutainment II. Springer, Berlin, pp 176–186

  25. Giannakopoulos G, Tatlas NA, Giannakopoulos V, Floros A, Katsoulis P (2018) Accessible electronic games for blind children and young people. Br J Educ Technol 49(4):608–619

    Article  Google Scholar 

  26. Hassenzahl M, Tractinsky N (2006) User experience-a research agenda. Behav Inf Technol 25(2):91–97

    Article  Google Scholar 

  27. Heuten W, Henze N, Boll S, Klante P (2007) Auditorypong–playing pong in the dark. Audio Mostly:134–147

  28. IJsselsteijn WA, De Kort YAW, Poels K (2013) The game experience questionnaire. Eindhoven: Technische Universiteit Eindhoven, pp 3–9

  29. Katz BF, Marentakis G (2016) Advances in auditory display research

  30. Khan N, Rahman AU (2018) Rethinking the mini-map: A navigational aid to support spatial learning in urban game environments. Int J Human–Comput Interaction 34(12):1135–1147

    Article  Google Scholar 

  31. Khowaja K, Salim SS (2019) Serious game for children with autism to learn vocabulary: an experimental evaluation. Int J Human–Comput Interaction 35(1):1–26

    Article  Google Scholar 

  32. Kim S, Lee KP, Nam TJ (2016) May). Sonic-badminton: audio-augmented badminton game for blind people. In: Proceedings of the 2016 CHI Conference extended abstracts on human factors in computing systems, pp. 1922–1929

  33. Lozano MD, Penichet VM, Leporini B, Fernando A (2018) Tangible user interfaces to ease the learning process of visually-impaired children. In: Proceedings of the 32nd International BCS human computer interaction conference, vol 32, pp 1–5

  34. Magnusson C, Rassmus-Gröhn K, Sjöström C, Danielsson H (2002) Navigation and recognition in complex haptic virtual environments–reports from an extensive study with blind users. In: Proceedings of Eurohaptics

  35. Merabet L, Sanchez J (2009) Audio-based navigation using virtual environments: combining technology and neuroscience. AER J Res Practice Vis Impairment Blindness 2(3):128–137

    Google Scholar 

  36. Morelli T, Foley J, Columna L, Lieberman L, Folmer E (2010) VI-Tennis: a vibrotactile/audio exergame for players who are visually impaired. In: Proceedings of the Fifth International Conference on the Foundations of Digital Games, pp 147–154

  37. Nakamura J, Csikszentmihalyi M (2009) Flow theory and research. Handb Positive Psychol:195–206

  38. Neto LV, Junior PHF, Bordini RA, Otsuka JL, Beder DM (2020) Design and implementation of an educational game considering issues for visually impaired people inclusion. Smart Learn Environ 7(1):1–16

    Article  Google Scholar 

  39. O’Donnell M (2002) Producing audio for halo. In: Game Developers Conference, San Jose. Available at: http://halo.bungie.org/misc/gdc

  40. Parseihian G, Gondre C, Aramaki M, Ystad S, Kronland-Martinet R (2016) Comparison and evaluation of sonification strategies for guidance tasks. IEEE Trans Multimed 18(4):674–686

    Article  Google Scholar 

  41. Rodríguez A, Boada I, Sbert M (2018) An Arduino-based device for visually impaired people to play videogames. Multimed Tools Appl 77 (15):19591–19613

    Article  Google Scholar 

  42. Sánchez J, Baloian N, Hassler T, Hoppe U (2003) Audiobattleship: blind learners collaboration through sound. In: CHI’03 Extended Abstracts on Human Factors in Computing Systems, pp 798–799

  43. Sánchez J, de Borba Campos M, Espinoza M, Merabet LB (2014) Audio haptic videogaming for developing wayfinding skills in learners who are blind. In: Proceedings of the 19th international conference on Intelligent user interfaces, pp. 199–208

  44. Sekhavat YA (2016) KioskAR: an augmented reality game as a new business model to present artworks. International Journal of Computer Games Technology 2016

  45. Sekhavat YA, Parsons J (2018) The effect of tracking technique on the quality of user experience for augmented reality mobile navigation. Multimed Tools Appl 77(10):11635–11668

    Article  Google Scholar 

  46. Sekhavat YA, Roohi S, Mohammadi HS, Yannakakis GN (2020) Play with One’s Feelings: A Study on Emotion Awareness for Player Experience. IEEE Transactions on Games

  47. Sekhavat YA, Sisi MJ, Roohi S (2021) Affective interaction: Using emotions as a user interface in games. Multimed Tools Appl 80(4):5225–5253

    Article  Google Scholar 

  48. Spagnol S, Saitis C, Bujacz M, Johannesson OI, Kalimeri K, Moldoveanu A, Unnthorsson R (2016) Model-based obstacle sonification for the navigation of visually impaired persons. In: Proceedings of the 19th International conference digital audio effects (DAFx-16). Brno, Czech Republic, pp 5–9

  49. Stewart J, Bauman S, Escobar M, Hilden J, Bihani K, Newman MW (2008) Accessible contextual information for urban orientation. In: Proceedings of the 10th international conference on Ubiquitous computing, pp 332–335

  50. Torres-Gil MA, Casanova-Gonzalez O, González-mora JL (2010) Applications of virtual reality for visually impaired people. WSEAS Trans Comput 9 (2):184–193

    Google Scholar 

  51. Von der Pütten AM, Klatt J, Ten Broeke S, McCall R, Krämer NC, Wetzel R, Klatt J (2012) Subjective and behavioral presence measurement and interactivity in the collaborative augmented reality game TimeWarp. Interact Comput 24(4):317–325

    Article  Google Scholar 

  52. Vorderer P, Wirth W, Gouveia FR, Biocca F, Saari T, Jäncke F, Klimmt C (2004) MEC spatial presence questionnaire (MEC-SPQ): Short documentation and instructions for application. Report to the European community, project presence: MEC (IST-2001-37661), pp 3

  53. Wall J (2002) Music for myst iii: Exile. the evolution of a videogame soundtrack

  54. Weibel D, Wissmath B (2011) Immersion in computer games: The role of spatial presence and flow. International Journal of Computer Games Technology

  55. Wersényi G (2009) Auditory representations of a graphical user interface for a better human-computer interaction. In: Auditory display. Springer, Berlin, pp 80–102

  56. Westin T (2004) Game accessibility case study: Terraformers–a real-time 3D graphic game In: Proceedings of the 5th International Conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT)

  57. Yuan B, Folmer E, Harris FC (2011) Game accessibility: a survey. Univ Access Inf Soc 10(1):81–100

    Article  Google Scholar 

Download references

Acknowledgements

This paper is financially supported by Tabriz Islamic Art University and University of Art based on the research project entitled “Studying the Capacities of Audio Arts in Computer Game Design for Visually Impaired Individuals”.

Author information

Authors and Affiliations

  1. Faculty of Multimedia, Tabriz Islamic Art University, Hakim Nezami sq, Azadi Blvd, Tabriz, Iran

    Yoones A. Sekhavat & Samad Roohi

  2. Faculty of Music, University of Art, Sarhang Sakhayi st, Tehran, Iran

    Mohammad Reza Azadehfar

  3. Institute for Cognitive & Brain Sciences, Shahid Beheshti University, Shahid Shahriari Square, Daneshjou Boulevard, Shahid Chamran Highway, Tehran, Iran

    Hossein Zarei

Authors
  1. Yoones A. Sekhavat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Reza Azadehfar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hossein Zarei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samad Roohi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yoones A. Sekhavat.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sekhavat, Y.A., Azadehfar, M.R., Zarei, H. et al. Sonification and interaction design in computer games for visually impaired individuals. Multimed Tools Appl 81, 7847–7871 (2022). https://doi.org/10.1007/s11042-022-11984-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11984-3

Keywords

Search

Navigation