Mobile Interactive Image Sonification for the Blind

  • Torsten Wörtwein
  • Boris Schauerte
  • Karin Müller
  • Rainer Stiefelhagen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9758)

Abstract

Web-based, mobile sonification offers a highly flexible way to give blind users access to graphical information and to solve various everyday as well as job-related tasks. The combination of a touch screen, image processing, and sonification allows the user to hear the content of every image region that he or she indicates with his/her finger position on a tablet or mobile phone. In this paper, we build on and expand our previous work in this area and evaluate how six blind participants can perceive mathematical graphs, bar charts, and floor maps with our sonifications and tactile graphics.

References

  1. 1.
    Banf, M., Blanz, V.: A modular computer vision sonification model for the visually impaired. In: Proceedings of the International Conference of Auditory Display (2012)Google Scholar
  2. 2.
    Banf, M., Blanz, V.: Sonification of images for the visually impaired using a multi-level approach. In: Proceedings of the Augmented Human International Conference (2013)Google Scholar
  3. 3.
    Bearman, N., Brown, E.: Who’s sonifying data and how are they ng it? A comparison of ICAD and other venues since 2009. In: Proceedings of the International Conference on Auditory Display (2012)Google Scholar
  4. 4.
    Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S., et al.: VizWiz: nearly real-time answers to visual questions. In: Proceedings of the Annual ACM Symposium on User Interface Software and Technology. pp. 333–342. ACM (2010)Google Scholar
  5. 5.
    Bigham, J.P., Jayant, C., Miller, A., White, B., Yeh, T.: VizWiz: locateit-enabling blind people to locate objects in their environment. In: Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 65–72 (2010)Google Scholar
  6. 6.
    Blüher, J.: About visorapps (2015). http://en.visorapps.com/about-visorapps/
  7. 7.
    Brooke, J.: SUS - a quick and dirty usability scale. Usability Eval. Ind. 189, 4–7 (1996)Google Scholar
  8. 8.
    Cavaco, S., Henriques, J.T., Mengucci, M., Correia, N., Medeiros, F.: Color sonification for the visually impaired. Procedia Technol. 9, 1048–1057 (2013)CrossRefGoogle Scholar
  9. 9.
    Grond, F., Droßard, T., Hermann, T.: SonicFunction: experiments with a function browser for the visually impaired. In: Proceedings of the International Conference on Auditory Display. Georgia Institute of Technology (2010)Google Scholar
  10. 10.
    Hermann, T., Hunt, A.: An introduction to interactive sonification. IEEE Multimedia 12, 20–24 (2005)CrossRefGoogle Scholar
  11. 11.
    Hermann, T., Hunt, A., Neuhoff, J.G.: The Sonification Handbook. Logos Verlag, Berlin (2011)Google Scholar
  12. 12.
    Heuten, W., Henze, N., Boll, S.: Interactive exploration of city maps with auditory torches. In: CHI Extended Abstracts on Human Factors in Computing Systems (2007)Google Scholar
  13. 13.
    Klatzky, R., Giudice, N., Bennett, C., Loomis, J.: Touch-screen technology for the dynamic display of 2D spatial information without vision: promise and progress. Multisensory Res. 27(5–6), 359–378 (2014)CrossRefGoogle Scholar
  14. 14.
    Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995)CrossRefGoogle Scholar
  15. 15.
    Martinez, M., Constantinescu, A., Schauerte, B., Koester, D., Stiefelhagen, R.: Cognitive evaluation of haptic and audio feedback in short range navigation tasks. In: Miesenberger, K., Fels, D., Archambault, D., Peňáz, P., Zagler, W. (eds.) ICCHP 2014, Part II. LNCS, vol. 8548, pp. 128–135. Springer, Heidelberg (2014)Google Scholar
  16. 16.
    Meijer, P.B.: An experimental system for auditory image representations. Trans. Biomed. Eng. 39(2), 112–121 (1992)CrossRefGoogle Scholar
  17. 17.
    Ribeiro, F., Florêncio, D., Chou, P.A., Zhang, Z.: Auditory augmented reality: object sonification for the visually impaired. In: Proceedings of the International Workshop on Multimedia Signal Processing (2012)Google Scholar
  18. 18.
    Rose, D.: Braille is spreading but who’s using it? (2013). http://www.bbc.com/news/magazine-16984742. bBCNews
  19. 19.
    Sanchez, J.: Recognizing shapes and gestures using sound as feedback. In: CHI Extended Abstracts on Human Factors in Computing Systems (2010)Google Scholar
  20. 20.
    Schauerte, B., Koester, D., Martinez, M., Stiefelhagen, R.: Way to Go! Detecting Open Areas Ahead of a Walking Person. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops, Part III. LNCS, vol. 8927, pp. 349–360. Springer, Heidelberg (2015)Google Scholar
  21. 21.
    Schauerte, B., Wörtwein, T., Stiefelhagen, R.: A web-based platform for interactive image sonification. In: Accessible Interaction for Visually Impaired People (2015)Google Scholar
  22. 22.
    Stevens, S.S., Volkmann, J., Newman, E.B.: A scale for the measurement of the psychological magnitude pitch. J. Acoust. Soc. Am. 8(3), 185–190 (1937)CrossRefGoogle Scholar
  23. 23.
    Su, J., Rosenzweig, A., Goel, A., de Lara, E., Truong, K.N.: Timbremap: enabling the visually-impaired to use maps on touch-enabled devices. In: Proceedings of the International Conference on Human Computer Interaction with Mobile Devices and Services (2010)Google Scholar
  24. 24.
    Taibbi, M., Bernareggi, C., Gerino, A., Ahmetovic, D., Mascetti, S.: AudioFunctions: eyes-free exploration of mathematical functions on tablets. In: Miesenberger, K., Fels, D., Archambault, D., Peňáz, P., Zagler, W. (eds.) ICCHP 2014, Part I. LNCS, vol. 8547, pp. 537–544. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  25. 25.
    Yoshida, T., Kitani, K.M., Koike, H., Belongie, S., Schlei, K.: EdgeSonic: image feature sonification for the visually impaired. In: Proceedings of the Augmented Human International Conference (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Torsten Wörtwein
    • 1
  • Boris Schauerte
    • 1
  • Karin Müller
    • 2
  • Rainer Stiefelhagen
    • 1
    • 2
  1. 1.Computer Vision for Human-Computer Interaction Lab (CVHCI)Karlsruhe Institute of TechnologyKarlsruheGermany
  2. 2.Study Centre for the Visually Impaired (SZS)Karlsruhe Institute of TechnologyKarlsruheGermany

Personalised recommendations