Using Computer Vision to See

  • Bogdan Mocanu
  • Ruxandra Tapu
  • Titus Zaharia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)


In this paper we propose a navigation assistant for visually impaired people, which uses computer vision techniques and is integrated on a wearable device. The system makes it possible to detect and recognize, in real-time, both static and dynamic objects existent in outdoor urban scenes without any a priori knowledge about the obstruction type or location. The detection system is based on relevant interest point extraction and tracking, background/camera motion estimation and foreground object identification through motion vectors clustering. The classification method receives as input image patches extracted by the detection module, performs global image representation using binary VLAD and prediction based on SVM. The feedback of our system is transmitted to visually impaired users through bone-conduction headphones as a set of audio warning messages. The entire system is fully integrated on a regular smartphone. The experimental evaluation performed on a set of 20 videos acquired with the help of VI users, demonstrates the pertinence of the proposed methodology.


Assistive wearable device Obstacle localization and recognition Acoustic feedback Visually impaired users 



This work was supported by a grant of the Romanian National Authority for Scientific Research and Innovation, CNCS - UEFISCDI, project number: PN-II-RU-TE-2014-4-0202.


  1. 1.
    Blasch, B.B., Wiener, W.R., Welsh, R.L.: Foundations of Orientation and Mobility, 2nd edn. American Foundation for the Blind, New York (1997)Google Scholar
  2. 2.
    Golledge, R.G., Marston, J.R., Costanzo, C.M.: Attitudes of visually impaired persons towards the use of public transportation. J. Vis. Impairment Blindness 90, 446–459 (1997)Google Scholar
  3. 3.
    Johnson, L.A., Higgins, C.M.: A navigation aid for the blind using tactile-visual sensory substitution. In: 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6289–6292 (2006)Google Scholar
  4. 4.
    Sainarayanan, G., Nagarajan, R., Yaacob, S.: Fuzzy image processing scheme for autonomous navigation of human blind. Appl. Soft Comput. 7(1), 257–264 (2007)CrossRefGoogle Scholar
  5. 5.
    Yu, J., Chung, H.I., Hahn, H.: Walking assistance system for sight impaired people based on a multimodal information transformation technique. In: ICCAS-SICE, pp. 1639–1643 (2009)Google Scholar
  6. 6.
    José, J., Farrajota, M., Rodrigues, J., Buf, J.D.: The smart vision local navigation aid for blind and visually impaired persons. Int. J. Digital Content Technol. Appl. 5, 362–375 (2011)Google Scholar
  7. 7.
    Lin, Q., Hahn, H., Han, Y.: Top-view based guidance for blind people using directional ellipse model. Int. J. Adv. Robot. Syst. 1, 1–10 (2013)Google Scholar
  8. 8.
    Peng, E., Peursum, P., Li, L., Venkatesh, S.: A smartphone-based obstacle sensor for the visually impaired. In: Yu, Z., Liscano, R., Chen, G., Zhang, D., Zhou, X. (eds.) UIC 2010. LNCS, vol. 6406, pp. 590–604. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Manduchi, R.: Mobile vision as assistive technology for the blind: an experimental study. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) ICCHP 2012, Part II. LNCS, vol. 7383, pp. 9–16. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Tapu, R., Mocanu, B., Bursuc, A., Zaharia, T.: A smartphone-based obstacle detection and classification system for assisting visually impaired people. In: IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 444–451 (2013)Google Scholar
  11. 11.
    Dakopoulos, D., Bourbakis, N.: Preserving visual information in low resolution images during navigation of visually impaired. In: Proceedings of the 1st International Conference on PErvasive Technologies Related to Assistive Environments, pp. 1–27 (2008)Google Scholar
  12. 12.
    Saez, J.M., Escolano, F., Penalver, A.: First steps towards stereo-based 6DoF SLAM for the visually impaired. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), p. 23 (2005)Google Scholar
  13. 13.
    Pradeep, V., Medioni, G., Weiland, J.: Robot vision for the visually impaired. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 15–22 (2010)Google Scholar
  14. 14.
    Saez, J.M., Escolano, F.: Stereo-based aerial obstacle detection for the visually impaired. In: Workshop on Computer Vision Applications for the Visually Impaired (2008)Google Scholar
  15. 15.
    Schauerte, B., Koester, D., Martinez, M., Stiefelhagen, R.: Way to go! detecting open areas ahead of a walking person. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014 Workshops. LNCS, vol. 8927, pp. 349–360. Springer, Heidelberg (2015)Google Scholar
  16. 16.
    Khan, A., Moideen, F., Lopez, J., Khoo, W.L., Zhu, Z.: KinDectect: kinect detecting objects. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) ICCHP 2012. LNCS, vol. 7383, pp. 588–595. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-31534-3_86 CrossRefGoogle Scholar
  17. 17.
    Takizawa, H., Yamaguchi, S., Aoyagi, M., Ezaki, N., Mizuno, S.: Kinect cane: an assistive system for the visually impaired based on three-dimensional object recognition. In: IEEE/SICE International Symposium on System Integration (SII), pp. 740–745 (2012)Google Scholar
  18. 18.
    Brock, M., Kristensson, P.: Supporting blind navigation using depth sensing and sonification. In: Proceedings of the ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 255–258 (2013)Google Scholar
  19. 19.
    Panteleris, P., Argyros, A.A.: Vision-based SLAM and moving objects tracking for the perceptual support of a smart walker platform. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 407–423. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16199-0_29 Google Scholar
  20. 20.
    Li, W., Li, X., Goldberg, M., Zhu, Z.: Face recognition by 3D registration for the visually impaired using a RGB-D sensor. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 763–777. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-16199-0_53 Google Scholar
  21. 21.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 589–600. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proceedings of Fourth Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  23. 23.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60, 63–86 (2004). Ubiquitous Intelligence and Computing SE - 45CrossRefGoogle Scholar
  24. 24.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, vol. 2, pp. 674–679 (1981)Google Scholar
  25. 25.
    Lee, J., Kim, G.: Robust estimation of camera homography using fuzzy RANSAC. In: Gervasi, O., Gavrilova, M.L. (eds.) ICCSA 2007. LNCS, vol. 4705, pp. 992–1002. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74472-6_81 CrossRefGoogle Scholar
  26. 26.
    Hamerly, G., Elkan, C.: Learning the k in k-means. In: Neural Information Processing Systems (2003)Google Scholar
  27. 27.
    Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)CrossRefGoogle Scholar
  28. 28.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  30. 30.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  31. 31.
    Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)CrossRefGoogle Scholar
  32. 32.
    Delhumeau, J., Gosselin, P.H., Jegou, H., Perez, P.: Revisiting the VLAD image representation. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 653–656 (2013)Google Scholar
  33. 33.
    Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graph. Stat. 15(2), 265–286 (2006)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Telecommunication Department, Faculty of ETTIUniversity “Politehnica” of BucharestBucharestRomania
  2. 2.ARTEMIS Department, Institut Mines-Telecom/Telecom SudParis, UMR CNRS MAP5 8145EvryFrance

Personalised recommendations