Abstract
In this paper, we present a novel wearable RGBD camera based navigation system for the visually impaired. The system is composed of a smartphone user interface, a glass-mounted RGBD camera device, a real-time navigation algorithm, and haptic feedback system. A smartphone interface provides an effective way to communicate to the system using audio and haptic feedback. In order to extract orientational information of the blind users, the navigation algorithm performs real-time 6-DOF feature based visual odometry using a glass-mounted RGBD camera as an input device. The navigation algorithm also builds a 3D voxel map of the environment and analyzes 3D traversability. A path planner of the navigation algorithm integrates information from the egomotion estimation and mapping and generates a safe and an efficient path to a waypoint delivered to the haptic feedback system. The haptic feedback system consisting of four micro-vibration motors is designed to guide the visually impaired user along the computed path and to minimize cognitive loads. The proposed system achieves real-time performance at \(28.4\)Hz in average on a laptop, and helps the visually impaired extends the range of their activities and improve the mobility performance in a cluttered environment. The experiment results show that navigation in indoor environments with the proposed system avoids collisions successfully and improves mobility performance of the user compared to conventional and state-of-the-art mobility aid devices.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Borenstein, J., Ulrich, I.: The GuideCane - A computerized travel aid for the active guidance of blind pedestrians. In: IEEE Int. Conf. on Robotics and Automation. pp. 1283–1288 (1997)
Brock, M., Kristensson, P.: Supporting blind navigation using depth sensing and sonification. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, pp. 255–258 (2013)
Clark-Carter, D., Heyes, A., Howarth, C.: The efficiency and walking speed of visually impaired people. Ergonomics 29(6), 779–789 (1986)
Golledge, R.G., Marston, J.R., Costanzo, C.M.: Attituds of visually imparied persons towards the use of public transportation. Journal of Visual Impairment Blindness 91(5), 446–459 (1997)
Google: Google glass (2013). http://www.google.com/glass
Guerrero, L., Vasquez, F., Ochoa, S.: An indoor navigation system for the visually impaired. Sensors 12(6), 8236–8258 (2012)
Helal, A., Moore, S., Ramachandran, B.: Drishti: An integrated navigation system for visually impaired and disabled. In: Proceedings Fifth International Symposium on Wearable Computers, pp. 149–156. IEEE (2001)
Hirschmuller, H., Innocent, P., Garibaldi, J.: Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics. In: 7th International Conference on Control, Automation, Robotics and Vision. ICARCV 2002. vol. 2, pp. 1099–1104. IEEE (2002)
Holzer, S., Rusu, R., Dixon, M., Gedikli, S., Navab, N.: Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2684–2689. IEEE (2012)
Horn, B.: Closed-form solution of absolute orientation using unit quaternions. JOSA A 4(4), 629–642 (1987)
Howard, A.: Real-time stereo visual odometry for autonomous ground vehicles. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2008, pp. 3946–3952. IEEE (2008)
Huang, A., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., Roy, N.: Visual odometry and mapping for autonomous flight using an rgb-d camera. In: International Symposium on Robotics Research (ISRR), pp. 1–16 (2011)
Koenig, S., Likhachev, M.: Fast replanning for navigation in unknown terrain. IEEE Transactions on Robotics 3(21), 354–363 (2005)
Lee, Y., Medioni, G.: A RGB-D camera based navigation for the visually impaired. In: RSS 2011 RGB-D: Advanced Reasoning with Depth Camera Workshop, pp. 1–6 (2011)
Legood, R., Scuffham, P., Cryer, C.: Are we blind to injuries in the visually impaired? a review of the literature. Injury Prevention 8(2), 155–160 (2002)
Loomis, J., Golledge, R., Klatzky, R.: Gps-based navigation systems for the visually impaired. Fundamentals of Wearable Computers and Augmented Reality 429(46) (2001)
Manduchi, R., Kurniawan, S.: Mobility-related accidents experienced by people with visual impairment. Insight: Research & Practice in Visual Impairment & Blindness 4(2), 44–54 (2011)
Martinez, J., Ruiz, F., et al.: Stereo-based aerial obstacle detection for the visually impaired. In: Workshop on Computer Vision Applications for the Visually Impaired (2008)
Mei, C., Sibley, G., Cummins, M., Newman, P., Reid, I.: A constant-time efficient stereo slam system. In: BMVC, pp. 1–11 (2009)
Moravec, H., Elfes, A.: High resolution maps from wide angle sonar. In: Proceedings 1985 IEEE International Conference on Robotics and Automation. vol. 2, pp. 116–121. IEEE (1985)
Nguyen, C., Izadi, S., Lovell, D.: Modeling kinect sensor noise for improved 3d reconstruction and tracking. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 524–530. IEEE (2012)
Pradeep, V., Medioni, G., Weiland, J.: Robot vision for the visually impaired. In: Computer Vision Applications for the Visually Impaired, pp. 15–22 (2010)
Sáez, J., Escolano, F.: 6dof entropy minimization slam for stereo-based wearable devices. Computer Vision and Image Understanding 115(2), 270–285 (2011)
Schiller, J., Peregoy, J.: Provisional report: Summary health statistics for u.s. adults: National health interview survey, 2011. National Center for Health Statistics 10(256) (2012)
JVIB news service: Demographics update: Alternate estimate of the number of guide dog users. Journal of Visual Impairment Blindness 89(2), 4–6 (1995)
Turano, K., Broman, A., Bandeen-Roche, K., Munoz, B., Rubin, G., West, S., SEE PROJECT TEAM: Association of visual field loss and mobility performance in older adults: Salisbury eye evaluation study. Optometry & Vision Science 81(5), 298–307 (2004)
West, S., Rubin, G., Broman, A., Munoz, B., Bandeen-Roche, K., Turano, K., THE SEE PROJECT: How does visual impairment affect performance on tasks of everyday life? Archives of Ophthalmology 120(6), 774–780 (2002)
Wurm, K., Hornung, A., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: A probabilistic, flexible, and compact 3d map representation for robotic systems. In: Proc. of the ICRA 2010 workshop on best practice in 3D perception and modeling for mobile manipulation. vol. 2 (2010)
Yguel, M., Aycard, O., Laugier, C.: Update policy of dense maps: Efficient algorithms and sparse representation. In: Field and Service Robotics. pp. 23–33. Springer (2008)
Yuan, D., Manduchi, R.: Dynamic environment exploration using a virtual white cane. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 243–249 (2005)
Zöllner, M., Huber, S., Jetter, H.-C., Reiterer, H.: NAVI – a proof-of-concept of a mobile navigational aid for visually impaired based on the microsoft kinect. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011, Part IV. LNCS, vol. 6949, pp. 584–587. Springer, Heidelberg (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Lee, Y.H., Medioni, G. (2015). Wearable RGBD Indoor Navigation System for the Blind. In: Agapito, L., Bronstein, M., Rother, C. (eds) Computer Vision - ECCV 2014 Workshops. ECCV 2014. Lecture Notes in Computer Science(), vol 8927. Springer, Cham. https://doi.org/10.1007/978-3-319-16199-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-16199-0_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16198-3
Online ISBN: 978-3-319-16199-0
eBook Packages: Computer ScienceComputer Science (R0)