Autonomous Identification of Virtual 3D Objects by Visually Impaired Users with Proprioception and Audio Feedback

  • Erico de Souza VeriscimoEmail author
  • João Luiz BernardesJr.
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9738)


The World Health Organization estimates that there are 285 million people with severe visual impairment worldwide. With the advent of technology, 3D virtual environments are being increasingly used for several applications. Many of these applications, however, are not accessible to visually impaired users, creating a digital divide. Our goal is to develop a novel and low cost 3D interaction technique to allow the identification of virtual objects with autonomy using only proprioception and hearing. We developed a prototype implementing this technique and conducted preliminary tests with it, first with sighted users, only to test the prototype’s basic functionalities and implementation, and then with a blind user with very promising results since the user was able to correctly and quickly identify virtual objects.


3D interaction Accessibility Visual impairment 


  1. 1.
    Lim, C.-K.: Application of kinect technology in the design of interactive products for chinese senior citizens. In: Stephanidis, C. (ed.) HCII 2013, Part I. CCIS, vol. 373, pp. 51–55. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  2. 2.
    Sousa, T., Cardoso, I., Parracho, J., Dias, P., Sousa Santos, B.: DETI-interact: interaction with large displays in public spaces using the kinect. In: Streitz, N., Markopoulos, P. (eds.) DAPI 2014. LNCS, vol. 8530, pp. 196–206. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  3. 3.
    Maike, V.R.M.L., Neto, L.S.B., Baranauskas, M.C.C., Goldenstein, S.K.: Seeing through the kinect: a survey on heuristics for building natural user interfaces environments in 8th international conference. In: UAHCI 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22–27, 2014, Proceedings, Part I (2014)Google Scholar
  4. 4.
    Rodriguez Esquivel, J.C., Meneses Viveros, A., Perry, N.: Gestures for interaction between the software CATIA and the human via microsoft kinect. In: Stephanidis, C. (ed.) HCI 2014, Part I. CCIS, vol. 434, pp. 457–462. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  5. 5.
    White, G., Fitzpatrick, G., McAllister, G.: Toward accessible 3D virtual environments for the blind and visually impaired. In: Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts. DIMEA 2008, vol. 349, pp. 134–141. ACM, New York (2008)Google Scholar
  6. 6.
    WHO, World Health Organization.: Global data on visual impairments 2010. Geneva, 17 p 2010. Web. 21 Nov. 2016.
  7. 7.
    UN, United Nations. Declaração de Direitos das Pessoas Deficientes in: Assembléia Geral da Organização das Nações Unidas. 09 Dez (1975)Google Scholar
  8. 8.
    Al-Khalifa, A.S., Al-Khalifa, H.S.: Do-It-Yourself object identification using augmented reality for visually impaired people. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) ICCHP 2012, Part II. LNCS, vol. 7383, pp. 560–565. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  9. 9.
    Nanayakkara, S., Shilkrot, R., Maes, P.: EyeRing: A finger-worn assistant. In: CHI 2012 Extended Abstracts on Human Factors in Computing Systems, 2012, pp. 1961–1966 (2012)Google Scholar
  10. 10.
    Nanayakkara S., Shilkrot, R.: EyeRing: a finger-worn input device for seamless interactions with our surroundings. In: AH 2013 Proceedings of the 4th Augmented Human International Conference (2013)Google Scholar
  11. 11.
    Veriscimo, E.S., Bernardes Júnior, J.L.: 3D Interaction accessible to visually impaired users: a systematic review in HCI International (2016)Google Scholar
  12. 12.
    Amiralian, M.L.T.M.: Compreendendo o cego. Uma visão psicanalítica da cegueira por meio de desenhos-estórias. São Paulo in Casa do Psicólogo, Fapesp (1997)Google Scholar
  13. 13.
    LEAP MOTION. Web. 06 Dez. 2014.
  14. 14.
  15. 15.

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Erico de Souza Veriscimo
    • 1
    Email author
  • João Luiz BernardesJr.
    • 1
  1. 1.School of Arts Sciences and Humanities – EACHUniversity of São PauloSão PauloBrazil

Personalised recommendations