An Assistive Vision System for the Blind That Helps Find Lost Things

  • Boris Schauerte
  • Manel Martinez
  • Angela Constantinescu
  • Rainer Stiefelhagen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7383)


We present a computer vision system that helps blind people find lost objects. To this end, we combine color- and SIFT-based object detection with sonification to guide the hand of the user towards potential target object locations. This way, we are able to guide the user’s attention and effectively reduce the space in the environment that needs to be explored. We verified the suitability of the proposed system in a user study.


Lost & Found Computer Vision Sonification Object Detection & Recognition Visually Impaired Blind 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amazon: Mechanical turk,
  2. 2.
    Bigham, J.P., Jayant, C., Miller, A., White, B., Yeh, T.: VizWiz:LocateIt - enabling blind people to locate objects in their environment. In: Proc. CVPR Workshop: Computer Vision Applications for the Visually Impaired (2010)Google Scholar
  3. 3.
    Caperna, S., Cheng, C., et al.: A navigation and object location device for the blind. Tech. rep., University of Maryland, College Park (2009)Google Scholar
  4. 4.
    Collet, A., Martinez, M., Srinivasa, S.S.: The MOPED framework: Object recognition and pose estimation for manipulation. Int. J. Robotics Research 30(10), 1284–1306 (2011)CrossRefGoogle Scholar
  5. 5.
    Constantinescu, A., Schultz, T.: Redundancy versus complexity in auditory displays for object localization - a pilot study. In: Proc. Int. Conf. Auditory Display (2011)Google Scholar
  6. 6.
    Durette, B., Louveton, N., Alleysson, D., Hrault, J.: Visuo-auditory sensory substitution for mobility assistance: Testing thevibe. In: Workshop on Computer Vision Applications for the Visually Impaired (2008)Google Scholar
  7. 7.
    Google Mobile: Google Goggles,
  8. 8.
    Hub, A., Diepstraten, J., Ertl, T.: Design and development of an indoor navigation and object identification system for the blind. In: Proc. Int. ACM SIGACCESS Conf. Computers and Accessibility (2004)Google Scholar
  9. 9.
    Schauerte, B., Fink, G.A.: Web-based learning of naturalized color models for human-machine interaction. In: Proc. Int. Conf. Digital Image Computing: Techniques and Applications (2010)Google Scholar
  10. 10.
    Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends. Comput. Graph. Vis. 3, 177–280 (2008)CrossRefGoogle Scholar
  11. 11.
    Wenqin, S., Wei, J., Jian, C.: A machine vision based navigation system for the blind. In: Proc. Int. Conf. Computer Science and Automation Engineering (2011)Google Scholar
  12. 12.
    Winlock, T., Christiansen, E., Belongie, S.: Toward real-time grocery detection for the visually impaired. In: Proc. CVPR Workshop: Computer Vision Applications for the Visually Impaired (2010)Google Scholar
  13. 13.
    World Health Organization: Visual impairment and blindness,

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Boris Schauerte
    • 1
  • Manel Martinez
    • 1
  • Angela Constantinescu
    • 2
  • Rainer Stiefelhagen
    • 1
    • 2
  1. 1.Institute for AnthropomaticsKarlsruhe Institute of TechnologyGermany
  2. 2.Study Center for the Visually Impaired StudentsKarlsruhe Institute of TechnologyKarlsruheGermany

Personalised recommendations