An Arduino-based device for visually impaired people to play videogames

Article
  • 353 Downloads

Abstract

Blind players have many difficulties to access video games since most of them rely on impressive graphics and immersive visual experiences. To overcome this limitation, we propose a device designed for visually impaired people to interact with virtual scenes of video games. The device has been designed considering usability, economic cost, and adaptability as main features. To ensure usability, we considered the white cane paradigm since this is the most used device by the blind community. Our device supports left to right movements and collision detection as well as actions to manipulate scene objects such as drag and drop. To enhance realism, it also integrates a library with sounds of different materials to reproduce object collision. To reduce the economic cost, we used Arduino as the basis of our development. Finally, to ensure adaptability, we created an application programming interface that supports the connection with different games engines and different scenarios. To test the acceptance of the device 12 blind participants were considered (6 males and 6 females). In addition, we created three mini-games in Unity3D that require navigation and walking as principal actions. After playing, participants filled a questionnaire related to usability and suitability to interact with games, among others. They scored well in all features without distinction among player gender and being blind from birth. The relationship between device responsiveness and user interaction has been considered satisfactory. Despite our small test sample, our main goal has been accomplished, the proposed device prototype seems to be useful to visually impaired people.

Keywords

Interacion devices Video games Visually impaired 

Notes

Acknowledgments

This work was supported by the Catalan Government (Grant No. 2014-SGR-1232) and by the Spanish Government (Grant No. TIN2016-75866-C3-3-R).

References

  1. 1.
    Abdel-Wahab AG, El-Masry AAA (2011) Mobile information communication technologies adoption in developing countries: effects and implications. Hershey, IGI GlobalCrossRefGoogle Scholar
  2. 2.
    Archambault D (2004) The TIM project: overview of results. In: Proc. Computers helping people with special needs, pp 248–256Google Scholar
  3. 3.
    Archambault D, Ossmann R, Gaudy T, Miesenberger K Computer games and visually impaired people (https://cedric.cnam.fr/fichiers/RC1204.pdf)
  4. 4.
    Arduino What is Arduino. Available online: https://www.arduino.cc/en/Guide/Introduction. Accessed 22 May 2017
  5. 5.
    Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadden L (1969) Vision substitution by tactile image projection. Nature 221:963–964CrossRefGoogle Scholar
  6. 6.
    Balan O, Moldoveanu A, Moldoveanu F (2015) Navigational audio games: an effective approach toward improving spatial contextual learning for blind people. Int J Disabil Hum Dev 14(2):109–118CrossRefGoogle Scholar
  7. 7.
    Brewster SA (1998) Using non-speech sounds to provide navigation cues. ACM Trans Comput-Human Interact 5:224–259CrossRefGoogle Scholar
  8. 8.
    Card SK, Robertson GG, Mackinlay JD (1991) The information visualizer: an information workspace. In: Proc. ACM CHI’91 Conf, 181–188Google Scholar
  9. 9.
    Csapó G, Nagy H, Stockman T (2015) A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research. J Multimodal User Interf 9(4):275–286CrossRefGoogle Scholar
  10. 10.
    Cui J, Liu Y, Xu Y, Zhao H, Zha H (2013) Tracking generic human motion via fusion of low- and high-dimensional approaches. IEEE Trans Syst Man Cybern Syst 43(4):996–1002CrossRefGoogle Scholar
  11. 11.
    Cuturi LF, Aggius-Vella E, Campus C, Parmiggiani A, Goria M (2016) From science to technology: orientation and mobility in blind children and adults. Neurosci Biobehav Rev 71:240–251CrossRefGoogle Scholar
  12. 12.
    Dabrowski J, Munson EV (2011) 40 Years of searching for the best computer system response time. Interact Comput 23:555–564CrossRefGoogle Scholar
  13. 13.
    Damaschini R, Legras R, Leroux R, Farcy R (2005) Electronic travel aid for blind people. Assistive Technol Virtual Realit 16(1):251–255Google Scholar
  14. 14.
    Farcy R, Leroux R, Damaschini R, Legras R, Bellik Y, Jacquet C, Pardo P (2003) Laser telemetry to improve the mobility of blind people: report of the 6 month training course. In: Proc. 1st International conference on smart homes and health telematics12, pp 113–115Google Scholar
  15. 15.
    Friberg J, Gärdenfors D (2004) Audio games: new perspectives on game audio. In: Proc. Int. ACM conference on advances in computer entertainment technology, pp 148–154Google Scholar
  16. 16.
    Ghali NI, Soluiman O, El-Bendary N, Nassef TM, Ahmed SA, Elbarawy YM, Hassanien AE (2012) Virtual reality technology for blind and visual impaired people: reviews and recent advances. In: Gulrez T, Hassanien A E (eds) Advances in robotics and virtual reality, ISRL 26, pp 363–385Google Scholar
  17. 17.
    Gori M, Cappagli G, Tonelli A, Baud-Bovy G, Finocchietti S (2016) Devices for visually impaired people: High technological devices with low user acceptance and no adaptability for children. Neurosci Biobehav Rev 69:79–88CrossRefGoogle Scholar
  18. 18.
    Gutschmidt R, Schiewe M, Zinke F, Jürgensen H (2010) Haptic emulation of games: haptic Sudoku for the blind. In: Proc. Int. Conf. on pervasive technologies related to assistive environments. Article 2. ACMGoogle Scholar
  19. 19.
    Hakobyan L, Lumsden J, OSullivan D, Bartlett H (2013) Mobile assistive technologies for the visually impaired. Surv Opthalmol 58(6):513–528CrossRefGoogle Scholar
  20. 20.
    Haptics rendering and applications Edited by Dr. Abdulmotaleb El Saddik, 978-953- 307-897-7 (2012)Google Scholar
  21. 21.
    Heuten W, Henze N, Boll S, Klante P (2007) Auditorypong, playing pong in the dark. In: Audio mostly- proc. conference on interaction with soundGoogle Scholar
  22. 22.
    Hossain E, Khan R, Muhida R, Ali A (2011) State of the art review on walking support system for visually impaired people. Int J Biomechatron Biomed Robot 1(4):232–251CrossRefGoogle Scholar
  23. 23.
    Kajimoto H, Inami M, Kawakami N, Tachi S (2003) Smart Touch-augmentation of skin sensation with electro cutaneous display, haptic interfaces for virtual environment and teleoperator systems. HAPTICS 2003. In: Proc. Symposium on IEEE, pp 40–46Google Scholar
  24. 24.
    Kay L (2000) Auditory perception of objects by blind persons, using a bioacoustic high resolution air sonar. J Acoust Soc Am 107:3266–3275CrossRefGoogle Scholar
  25. 25.
    Kim J, Ricaurte J (2011) TapBeats: accessible and mobile casual gaming. In: Proc. ACM Conference on computers and accessibility. ACM, pp 285–286Google Scholar
  26. 26.
    Kim S, Lee K, Nam T (2016) Sonic-badminton: audio-augmented badminton game for blind people. In: Proc. Conference on human factors in computing systems. ACM, pp 1922–1929Google Scholar
  27. 27.
    Lacey G, Dawson-Howe KM, Vernon D (1995) Personal autonomous mobility aid for the frail and elderly blind (Technical Report, No. TCD-CS-95-18). Trinity College, DublinGoogle Scholar
  28. 28.
    Leonard R (2002) Statistics on vision impairment: a resource manual, 5th edn. Arlene Gordon Research Institute of Lighthouse International, New YorkGoogle Scholar
  29. 29.
    Liu Y, Zhang X, Cui J (2010) Visual analysis of child-adult interactive behaviors in video sequences. Int Conf Virt Syst MultimedGoogle Scholar
  30. 30.
    Liu Y, Cui J, Zhao H, Zha H (2012) Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking. Int Conf Pattern Recogn 898–901Google Scholar
  31. 31.
    Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: recognizing complex activities from sensor data. In: Yang Q, Wooldridge M (eds) Proc. int. conference on artificial intelligence. AAAI Press, pp 1617–1623Google Scholar
  32. 32.
    Liu L, Cheng L, Liu Y, Jia Y, Rosenblum DS (2016) Recognizing complex activities by a probabilistic interval-based model. Int Conf Artific Intell, 1266–1272Google Scholar
  33. 33.
    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) tFortune teller: predicting your career path. Int Conf Artif Intell, 201–207Google Scholar
  34. 34.
    Liu Y, Nie L, Liu L, Rosenblum D S (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181(2016):108–115CrossRefGoogle Scholar
  35. 35.
    Lu Y, Wei Y, Liu L, Zhong J, Sun L, Liu Y (2017) Towards unsupervised physical activity recognition using smartphone accelerometers. Multimed Tools 76(8):10701–10719CrossRefGoogle Scholar
  36. 36.
    Maidenbaum S, Levy-Tzedek S, Chebat D, NamerFurstenberg R, Amedi A (2014) The effect of expanded sensory range via the eyecane sensory substitution device on the characteristics of visionless virtual navigation,MSR. ISSN 2213–4794Google Scholar
  37. 37.
    Manduchi R, Coughlan JM (2012) Computer vision without sight. Commun ACM 55(1):96–104CrossRefGoogle Scholar
  38. 38.
    Manduchi R, Kurniawan S (2011) Mobility-related accidents experienced by people with visual impairment. Res Pract Vis Impairment Blindness 4(2):44–54Google Scholar
  39. 39.
    Manduchi R, Kurniawan S (2012) Assistive technology for blindness and low vision. CRC Press, ISBN 9781439871539Google Scholar
  40. 40.
    Magnusson C, Rassmus-Grhn K, Sjstrm C, Danielsson H (2002) Navigation and recognition in complex haptic virtual environments? Reports from an extensive study with blind users? In: Proc. Eurohaptics 2002. EdinburghGoogle Scholar
  41. 41.
    Mau S, Melchior NA, Makatchev M, Steinfeld A (2008) BlindAid: an electronic travel aid for the blind (Technical Report, No. CMU-RI-TR-07-39). The Robotics Institute at Carnegie Mellon University, PittsburghGoogle Scholar
  42. 42.
    Miller RB (1968) Response time in man-computer conversational transactions. Proc AFIPS Fall Joint Comput Conf 33:267–277Google Scholar
  43. 43.
    Miller D, Parecki A, Douglas S (2007) Finger dance: a sound game for blind people. In: Proc. Int. ACM conference on computers and accessibility, pp 253–254Google Scholar
  44. 44.
    Milne L, Bennett C, Ladner R, Azenkot S (2014) Brailleplay: educational smartphone games for blind children. In: Proc. Int. conference on computers and accessibility, pp 137–144Google Scholar
  45. 45.
    Morelli T, Foley J, Columna L, Lieberman L, Folmer E (2010) VI-Tennis a vibrotactile / audio exergame for players who are visually impaired categories and subject descriptors. In: Proc. Int. ACM conference on the foundations of digital games, pp 147–154Google Scholar
  46. 46.
    Morelli T, Foley J, Folmer E (2010) Vi-bowling: a tactile spatial exergame for individuals with visual impairments. In: Proc. International conference on computers and accessibility, pp 179–186Google Scholar
  47. 47.
    Nagarajan R, Yaacob S, Sainarayanan G (2003) Role of object identification in sonification system for visually impaired. In: IEEE Tencon (IEEE region 10 conference on convergent technologies for the AsiaPacific). Bangalore, pp 15–17Google Scholar
  48. 48.
    Neilson J (1993) Usability engineeringGoogle Scholar
  49. 49.
    Nikolakis G, Tzovaras D, Moustakidis S, Strintzis M (2004) Cybergrasp and phantom integration: Enhanced haptic access for visually impaired users. Conf Speech Comput 20–22Google Scholar
  50. 50.
    Papa Sangre (2013) http://www.papasangre.com/
  51. 51.
    Rempel J (2012) Glasses that alert travelers to objects through vibration? An evaluation of iGlasses by RNIB and AmbuTech. AFB Access World Mag 13:9Google Scholar
  52. 52.
    Revuelta P, Ruiz B, Sánchez JM, Bruce N (2014) Scenes and images into sounds: a taxonomy of image sonification methods for mobility applications. J Audio Eng Soc 62(3):161–171CrossRefGoogle Scholar
  53. 53.
    Rodriguez A, Garcia RJ, Garcia JM, Magdics M, Sbert M (2013) Implementation of a videogame: legends of girona actas del primer simposio espaol de entretenimiento digital. In: Gonzlez P A, Gmez, M A (eds) pp 96–107Google Scholar
  54. 54.
    Rodriguez-Sancheza MC, Moreno-Alvareza MA, Martin E, Borromeoa S, Hernandez-Tamamesa JA (2014) Accessible smartphones for blind users: a case study for a way finding system. Expert Syst Appl 41(16):7210–7222CrossRefGoogle Scholar
  55. 55.
    Roentgen UR, Gelderblom GJ, Soede M, de Witte L (2008) Inventory of electronic mobility aids for persons with visual impairments: a literature review. J Vis Impairment Blindness 102(11):702– 724Google Scholar
  56. 56.
    Sakhardande J, Pattanayak P, Bhowmick M (1163) Arduino based mobility caneinternational. J Sci Eng Res 4:4Google Scholar
  57. 57.
    Sánchez J, Sáenz M, Ripoll M (2009) Usability of a multimodal videogame to improve navigation skills for blind children. In: Proc. ACM Computers and accessibility, pp 35–42Google Scholar
  58. 58.
    Sánchez J, Sáenz J, Garrido JM (2010) Usability of a multimodal video game to improve navigation skills for blind children. TACCESS 3:2010CrossRefGoogle Scholar
  59. 59.
    Sánchez J, Campos M, Espinoza M, Merabet LB (2014) Audio haptic video gaming for developing way finding skills in learners who are blind. In: Proc. International conference on intelligent user interfaces. ACM, pp 199–208Google Scholar
  60. 60.
    Savidis A, Stamou A, Stephanidis C (2007) An accessible multimodal pong game space. Universal Access Ambient Intell Environ 405–418Google Scholar
  61. 61.
    Shoval S, Borenstein J, Koren Y (1998) The Navbelt-A computerized travel aid for the blind based on mobile robotics technology. IEEE Trans Biomed Eng 45:1376–1386CrossRefGoogle Scholar
  62. 62.
    Sohl-Dickstein J, Teng S, Gaub BM, Rodgers CC, Li C, DeWeese MR, Harper NS (2015) A device for human ultrasonic echolocation. IEEE Trans Biomed Eng 62:1526–1534CrossRefGoogle Scholar
  63. 63.
    Sudhanthiradevi M, Suganya Devi M, Roshini R, Sathya T (2016) Arduino based walking stick for visually impaired. Int J Adv Res Trends Eng Technol 5:4Google Scholar
  64. 64.
    Sung-Yeon K, Kwangsu C (2013) Usability and design guidelines of smart canes for users with visual impairments. Int J Des 7:1Google Scholar
  65. 65.
    Torres-Gil MA, Casanova-Gonzalez O, Gonzalez-Mora JL (2010) Applications of virtual reality for visually impaired people. WSEAS Trans Comput 2(9):184–193Google Scholar
  66. 66.
    Tzovaras D, Moustakas K, Nikolakis G, Strinzis M (2009) Interactive mixed reality white cane simulation for the training of the blind and the visually impaired. Person Ubiq Comput 13(1):51–58CrossRefGoogle Scholar
  67. 67.
    Velzquez R (2010) Wearable assistive devices for the blind. Wearable and autonomous biomedical devices and systems for smart environment. Lect Notes Electr Eng 75:331–349CrossRefGoogle Scholar
  68. 68.
    Vorderer P, Bryant J (2006) Playing video games-motives, responses and consequences, Mahwab, NJ:LEA, ISBN: 978-0805853223Google Scholar
  69. 69.
    Wood J, Magennis M, Cano Arias EF, Gutierrez T, Graupp H, Bergamasco M (2003) The design and evaluation of a computer game for the blind in the GRAB haptic audio virtual environment eurohaptics 2003. DublinGoogle Scholar
  70. 70.
    www.theesa.com/. Accessed on May 2017
  71. 71.
    Yuan B, Folmer E (2008) Blind hero: enabling guitar hero for the visually impaired. In: Proc. International ACM conference on computers and accessibility, pp 169–176Google Scholar
  72. 72.
    Yuan B, Folmer E, Harris FC (2011) Game accessibility: a survey. Univers Access Inf Soc 10(1):81–100CrossRefGoogle Scholar
  73. 73.
    Zhao S, Yao H, Gao Y, Ji R, Xie W, Jiang X, Chua TS (2016) Predicting personalized emotion perceptions of social images. Proc ACM Multimed Conf 1385–1394Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Graphics and Imaging LaboratoryUniversity of GironaGironaSpain
  2. 2.Department of Computer Science and TechnologyTianjinChina

Personalised recommendations