Machine Vision and Applications

, Volume 24, Issue 3, pp 521–535 | Cite as

Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments

  • YingLi TianEmail author
  • Xiaodong Yang
  • Chucai Yi
  • Aries Arditi
Original Paper


Independent travel is a well-known challenge for blind and visually impaired persons. In this paper, we propose a proof-of-concept computer vision-based wayfinding aid for blind people to independently access unfamiliar indoor environments. In order to find different rooms (e.g. an office, a laboratory, or a bathroom) and other building amenities (e.g. an exit or an elevator), we incorporate object detection with text recognition. First, we develop a robust and efficient algorithm to detect doors, elevators, and cabinets based on their general geometric shape, by combining edges and corners. The algorithm is general enough to handle large intra-class variations of objects with different appearances among different indoor environments, as well as small inter-class differences between different objects such as doors and door-like cabinets. Next, to distinguish intra-class objects (e.g. an office door from a bathroom door), we extract and recognize text information associated with the detected objects. For text recognition, we first extract text regions from signs with multiple colors and possibly complex backgrounds, and then apply character localization and topological analysis to filter out background interference. The extracted text is recognized using off-the-shelf optical character recognition software products. The object type, orientation, location, and text information are presented to the blind traveler as speech.


Indoor wayfinding Computer vision Object detection Text extraction Optical character recognition (OCR) Blind/visually impaired persons 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arditi, A., Brabyn, J.: Signage and wayfinding. In: Silverstone, B., Lang, M.A., Rosenthal, B., Faye, E. (eds.) The Lighthouse Handbook on Visual Impairment and Vision Rehabilitation, Oxford University Press, New York (2000)Google Scholar
  2. 2.
    Anguelov, D., Koller, D., Parker, E., Thrun, S.: Detecting and modeling doors with mobile robots. In: Proceedings of the IEEE international conference on robotics and automation (2004)Google Scholar
  3. 3.
    Baker, A.: Blind Man is Found Dead in Elevator Shaft. The New York Times, City Room (2010)Google Scholar
  4. 4.
    Biederman, I.: Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94 (1987)Google Scholar
  5. 5.
    Blind Sight: A camera for visually impaired people.
  6. 6.
    Canny J.: A computational approach to edge detection. IEEE Trans. Pattern Analy. Mach. Intell. PAMI 8, 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Chen, X., Yuille, A.: Detecting and reading text in natural scenes, CVPR (2004)Google Scholar
  8. 8.
    Chen, Z., Birchfield, S.: Visual detection of lintel-occluded doors from a single image. IEEE Computer Society workshop on visual localization for mobile platforms (2008)Google Scholar
  9. 9.
    Chen, C., Tian, Y.: Door detection via signage context-based hierarchical compositional model. 2nd workshop on use of context in video processing (UCVP) (2010)Google Scholar
  10. 10.
    Dakopoulos D., Bourbakis N.G.: Wearable obstacle avoidance electronic travel aids for blind: a survey. IEEE IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(1), 25–35 (2010)CrossRefGoogle Scholar
  11. 11.
    Dinh, V., Chun, S., Cha, S., Ryu, H., Sull, S.: An efficient method for text detection in video based on stroke width similarity. Asian conference on computer vision (ACCV) (2007)Google Scholar
  12. 12.
    Divvala, S., Hoiem, D., Hays, J., Efros, A., Hebert, M.: An empirical study of context in object detection. In: Proceedings of IEEE CVPR (2009)Google Scholar
  13. 13.
    Dubey, P.: Edge based text detection for multi-purpose application. Int. Conf. Signal Process. 4 (2006)Google Scholar
  14. 14.
    Everingham, M., Thomas, B., Troscianko, T.: Wearable mobility aid for low vision using scene classification in a Markov random field model framework. Int. J. Hum. Comput. Interact. 15(2) (2003)Google Scholar
  15. 15.
    Giudice N., Legge G.: Blind navigation and the role of technology. In: Helal, A.A., Mokhtari, M., Abdulrazak, B. (eds) The engineering handbook of smart technology for aging, disability, and independence., Wiley, Hoboken (2008)Google Scholar
  16. 16.
    He, X., Yung, N.: Corner detector based on global and local curvature properties. Opt. Eng. 47(5) (2008)Google Scholar
  17. 17.
    Hensler, J., Blaich, M., Bittel, O.: Real-time door detection based on adaboost learning algorithm. International conference on research and education in robotics, Eurobot (2009)Google Scholar
  18. 18.
    Ivanchenko, V., Coughlan J., Shen, H.: Crosswatch: a camera phone system for orienting visually impaired pedestrians at traffic intersections. 11th international conference on computers helping people with special needs (ICCHP ’08) (2008)Google Scholar
  19. 19.
    Kasar, T., Kumar, J., Ramakrishnan, A.G.: Font and background color independent text binarization. Second international workshop on camera-based document analysis and recognition (2007)Google Scholar
  20. 20.
    Kim, D., Nevatia, R.: A method for recognition and localization of generic objects for indoor navigation. In: ARPA image understanding workshop (1994)Google Scholar
  21. 21.
    Kreiman, G.: Biological object recognition. Scholarpedia 3(6), 2667. (2008)
  22. 22.
    Liu, C., Wang, C., Dai, R.: Text detection in images based on unsupervised classification of edge-based features. International conference on document analysis and recognition (2005)Google Scholar
  23. 23.
    Liu, Q., Jung, C., Moon, Y.: Text segmentation based on stroke filter. In: Proceedings of international conference on multimedia (2006)Google Scholar
  24. 24.
    Luo, J., Singhal, A., Zhu, W.: Natural object detection in outdoor scenes based on probabilistic spatial context models. International conference on multimedia and expo (2003)Google Scholar
  25. 25.
    Manduchi, R., Coughlan, J., Ivanchenko, V.: Search strategies of visually impaired persons using a camera phone wayfinding system. 11th international conference on computers helping people with special needs (ICCHP ’08) (2008)Google Scholar
  26. 26.
    Munoz-Salinas, R., Aguirre, E., Garcia-Silvente, M., Gonzalez, A.: Door-detection using computer vision and fuzzy logic. In: Proceedings of the 6th WSEAS international conference on mathematical methods and computational techniques in electrical engineering (2004)Google Scholar
  27. 27.
    Murillo, A., Kosecka, J., Guerrero, J., Sagues, C.: Visual door detection integrating appearance and shape cues. Robot. Auton. Syst. (2008)Google Scholar
  28. 28.
    National Research Council. Electronic travel aids: new directions for research. Working group on mobility aids for the visually impaired and blind, ed. C.o. vision. National Academy Press, Washington, DC, p. 107 (1986)Google Scholar
  29. 29.
    Nikolaou, N., Papamarkos, N.: Color reduction for complex document images. Int. J. Imaging Syst. Technol. 19 (2009)Google Scholar
  30. 30.
    Oliva A., Torralba A.: The role of context in object recognition. Trends Cognit. Sci. 11, 520–527 (2007)CrossRefGoogle Scholar
  31. 31.
    Paletta, L., Greindl, C.: Context based object detection from video. In: Proceedings of international conference on computer vision systems (2003)Google Scholar
  32. 32.
    Pradeep, V., Medioni, G., Weiland, J.: Piecewise planar modeling for step detection using stereo vision. Workshop on computer vision applications for the visually impaired (2008)Google Scholar
  33. 33.
    Shen, H., Coughlan, J.: Grouping using factor graphs: an approach for finding text with a camera phone. Workshop on graph-based representations in pattern recognition (2007)Google Scholar
  34. 34.
    Shivakumara, P., Huang, W., Tan, C.: An efficient edge based technique for text detection in video frames. The eighth IAPR workshop on document analysis systems (2008)Google Scholar
  35. 35.
    Stoeter, S., Mauff, F., Papanikolopoulos, N.: Realtime door detection in cluttered environments. In: Proceedings of the 15th IEEE international symposium on intelligent control (2000)Google Scholar
  36. 36.
    Tian, Y., Yi, C., Arditi, A.: Improving computer vision-based indoor wayfinding for blind persons with context information. 12th international conference on computers helping people with special needs (ICCHP) (2010)Google Scholar
  37. 37.
    Tian, Y., Yang, X., Arditi, A.: Computer vision-based door detection for accessibility of unfamiliar environments to blind persons. 12th international conference on computers helping people with special needs (ICCHP) (2010)Google Scholar
  38. 38.
    Torralba A.: Contextual priming for object detection. Int. J. Comput. Vision 53(2), 169–191 (2003)CrossRefGoogle Scholar
  39. 39.
    Tran, H., Lux, A., Nguyen, H., Boucher, A.: A novel approach for text detection in images using structural features. The 3rd international conference on advances in pattern recognition (2005)Google Scholar
  40. 40.
    Seeing with sound—the vOICe.
  41. 41.
    Wan, M., Zhang, F., Cheng, H., Liu, Q.: Text localization in spam image using edge features. International conference on communications, circuits and system (2008)Google Scholar
  42. 42.
    Wong, E., Chen, M.: A new robust algorithm for video text extraction. Pattern Recognit. 36 (2003)Google Scholar
  43. 43.
    Yang, X., Tian, Y.: Robust door detection in unfamiliar environments by combining edge and corner features. 3rd workshop on computer vision applications for the visually impaired (CVAVI) (2010)Google Scholar
  44. 44.
    Zandifar, A., Duraiswami, R., Chahine, A., Davis, L.: A video based interface to textual information for the visually impaired. In: Proceedings of IEEE 4th international conference on multimodal interfaces (2002)Google Scholar
  45. 45.

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • YingLi Tian
    • 1
    Email author
  • Xiaodong Yang
    • 2
  • Chucai Yi
    • 3
  • Aries Arditi
    • 4
  1. 1.Electrical Engineering DepartmentThe City College, and Graduate Center, City University of New YorkNew YorkUSA
  2. 2.Electrical Engineering DepartmentThe City College, City University of New YorkNew YorkUSA
  3. 3.The Graduate CenterCity University of New YorkNew YorkUSA
  4. 4.Visibility Metrics LLCChappaquaUSA

Personalised recommendations