Keywords

1 Introduction

Globally, it is estimated that there are 285 million people who are blind or visually impaired [1]. For many, traveling to unfamiliar environments is one of the most difficult challenges for visually impaired people [7]. Although numerous studies have attempted to address such navigation challenges by using a smartphone, mobile robots, etc., still they do not adequately address the needs of people who are blind or visually impaired due to limited accuracy, accessibility, and adaptability. For example, blind travelers may not fully depend on the existing indoor navigation technologies because it would include 1–2 m localization error [2]. Because of such issues, the blind often participates in mobility training sessions with O&M (Orientation and Mobility) instructors in order to become familiar with a new place. However, such training is time consuming and expensive. Furthermore, it is often challenging for the blind to memorize all the steps necessary to independently navigate in a new location [6].

The objective of this study is to develop new, low-cost, and accessible navigation aid tools that do not employ any localization nor indoor positioning techniques and not require a direct interaction and training with O&M instructors for blind and visually impaired people to travel safely and independently in unfamiliar environments. To this end, we propose a system that generates optimized paths and very detailed instructions on a computer installed at an indoor kiosk and wirelessly transfers the generated instructions to the user’s smartphone. The paths and the instructions are generated based on a shoreline approach that is a strategy used by O&M instructors when they teach independent navigation skills to people who are visually impaired. With the shoreline approach, blind people could continuously follow recognizable landmarks such as walls, doors, and intersections with his or her traditional white cane until they reach their destination. The judgment of the arrival of each landmark is the responsibility of the visually impaired, and accuracy of judgment can be improved by providing an easily recognizable landmark to the visually impaired. We envision this system being available in public spaces, more specifically at indoor kiosks to assist those needing help with navigation (like an audio guide tour system in museums).

This paper consists of 5 sections, including this introductory section. The rest of the paper is organized as follows. In Sect. 2, we present related works to this study including indoor positioning techniques and smartphone-based navigation systems. We then described the proposed system in Sect. 3. In Sect. 4, the user study procedures conducted, results and analysis are included. Section 5 summarizes the conclusions and future scope of this study.

2 Related Work

Numerous researchers have attempted to find adequate navigation aids for blind and visually impaired people. These related assistive technologies include a wide range of navigation tools such as devices utilizing indoor positioning techniques (Wi-Fi fingerprinting [8, 14] and dead reckoning [3, 11]), physical infrastructure in the building (RFID tags) [12], or mobile devices, for example multiple sensors, smartphones [13], and wearable devices [4, 9].

Since GPS (Global Positioning System) is unavailable in the indoor situation, the indoor localization techniques mentioned above have been widely used to keep track the user’s position. Wi-Fi fingerprinting is the system that utilizes wireless access points (APs) to obtain the location information. On the other hand, dead reckoning is another indoor localizing method that more relies on the computation, which calculates the estimated position with the velocity and starting point of the user. Also, some research attempted to combine these techniques together for enhancing the route planning algorithm effectively [8]. However, Wi-Fi fingerprinting is still inaccurate to calculate the position of the user. Additionally, several studies that addressed the dead reckoning showed several accuracy issues because this localization technique requires the computational overhead during traveling and the error in this method easily grows as time passes [3].

A smartphone takes advantage of flexibility and functionality due to its accessibility. However, most research using the smartphone concentrated on the outdoor navigation [13] or had relatively lower accuracy in indoor situations [5]. Those systems are creative and effective for sighted people, but they are not suitable for blind people for their reliable navigation, because the suggested systems entail a considerable error (1–2 m) and low success rates which make users get confused and sometimes frustrated. For the higher accuracy, another study installed the physical infrastructure such as RFID [12]. Its main drawback is the high cost and long time taken to install these tags in the building.

Previous research has certain limitations such as error-prone systems, additional infrastructure, and inconvenience. Therefore, it is more restricted to a constrained use. On the other hand, our work involves the use of simplicity for the users in terms of a form factor, size, usability, robustness, and safety. Also we use a smartphone along with three simple gestures (left/right swipe, double tap) to achieve a navigation device capable of being held in hand or working with the user’s cane.

Fig. 1.
figure 1

Overall architecture of the system - The server contains two user interfaces which are for authorized users and visually impaired users. The server and smartphone communicate via Bluetooth. The smartphone receives generated instructions. Depending on a user input, it plays the instructions step by step using TTS.

Full size image

3 System Design

The overall architecture of the navigation system is depicted in Fig. 1, and the user interface is shown in Fig. 2(a). The software is divided into two parts. One part is designed for authorized users such as building managers and O&M instructors to create a database on the interesting environment [10]. The traditional way to instruct blind people to navigate new paths is that instructors usually explore the requested location and find features around the routes in advance. They then teach the determined paths, which is finalized by many simulations. Likewise, this software is operated to add potential features by authorized users, which can be stored in the database and used for the path planning. This concept can update a software and the path planning in real-time when authorized users enter a new landmark. Based on the collected information, the system creates a directed graph. A tuple indicates a node of the directed graph, and tuples are connected with the near feature (edge of the graph). Each edge has various weights in order to reflect difficulties in finding the next landmark at the current position. For example, following an adjacent wall is much easier than crossing to the hall way. In this case, when a user needs to cross the hall way, the edge has a higher cost than just following an adjacent wall. Thus, the weights are different depending on types of current feature and the next node.

Fig. 2.
figure 2

User interface for the software.

Full size image

Another part of the software is designed for the blind or visually impaired who mainly uses this software in a real-world situation. When a user comes to a kiosk in the building, the user will be required to enter their destination. After receiving the destination from users, the software generates the optimized paths to the destination using a shoreline-based path planning. The generated paths come from the directed graph which was generated by the first part of the software, and the software uses a Dijkstra algorithm for path planning. In order to give a detailed instruction, the system provides a direction, a distance, and the number of doors that a user has to pass. An example of the generated instruction is “Go straight until the corner. The distance is almost 15 ft. And you will pass 2 doors.”

Last, we developed an Android App that enables users to listen to voice instructions while traveling with their existing navigation aid tools such as a white cane. Once smartphone receives the instructions generated by the kiosk, the App then (Fig. 2(b)) plays them by converting into speech using TTS. In addition, considering the interaction factors, the user can simply operate the App by three simple gestures (left/right swipe and double tap). Each gesture plays the next, previous, and current instruction. With this proposed system, users do not need to memorize the entire paths in order to reach their destination. The arrival at each landmark is determined by the user, and if the user determines that he or she has reached the landmark, the user plays the next instruction to move to the next landmark.

Fig. 3.
figure 3

User study - Three visually impaired participants navigating unfamiliar indoor environments with the help of the proposed system.

Full size image

4 User Study

In order to evaluate the feasibility and usefulness of the system, we recruited three blind and visually impaired participants as shown in Fig. 3 (P1, P3 - totally blind, P2- partially blind who can potentially recognize lights, all males)Footnote 1. Before starting the actual experiment, each participant participated in the two tutorial sessions which made users comfortably operate the device and Android App.

Then, the participants conducted three predefined tasks that consist of different difficulty levels (See Fig. 4). The total travel distance of each path increases as the difficulty level gets higher. After finishing the tasks, we got user feedback from each participant. The questions (Table 1) are related to perceived usefulness and ease of use of the system.

4.1 Results and Analysis

Time and Trajectory. Investigators measured the time (Table 2) and recorded trajectories while the participants were traveling from the starting point to the destination. Overall, each participant followed the tutorial sessions safely and finished the three tasks well. One remarkable observation is that each participant used the smartphone in a totally different way. P1 played instructions step by step as we expected. On other hand, P2 put the smartphone in his pocket and played one by one. For a right handed person, it is hard to carry both a white cane and the smartphone at the same time. It shows that P2 took a long time to finish the first task, but he finished the other two tasks much faster than the other two participants in the task 2. In addition, P3 played all instructions at once and created a virtual map in his mind. Because of that, P3 took a relatively longer time to arrive at the destination. However, most of the time he spent was when he was playing all the instructions, and therefore the actual travel time was the fastest among all the participants. It was also observed that some participants had trouble with finding landmarks (particularly doors). Due to this issue, P1 failed to find the destination in the task 2 although he was able to move around the destination, and P3 had to repeat the previous instructions and correct his position. Figure 5 shows the trajectories that P1 and P2 explored. All participants did not deviate the desired paths over 3 m, and they were confident that they could arrive at the right destination. This result shows that blind and visually impaired users can arrive in unfamiliar spaces with the proposed system.

Experiment video is available at: https://goo.gl/VARGvY.

Fig. 4.
figure 4

Map representation and numerical information of predefined tasks.

Full size image
Table 1. Survey questions.
Full size table
Fig. 5.
figure 5

Trajectory results of the user study.

Full size image
Table 2. Travel time - Only P1 failed to find a destination during Task 2.
Full size table
Table 3. Survey results - We used Likert scale (1: Strongly disagree, 2: Disagree, 3: Neutral, 4: Agree, 5: Strongly agree).
Full size table

User Feedback. Most of them evaluated the system as useful and easy to use (see Table 3). All the participants strongly agreed that the proposed system would help visually impaired people travel a new place, and the generated instructions helped them reach their destinations. Therefore, they all recommend this device.

Q1 and Q3 scored relatively lower than other questions. This system mainly utilizes the walls in the building. Thus, the participants delineated some situations when they need to travel an open space, which does not include any wall. Also, they suggested potential improvements of the system. For example, if the user gets lost, the only way to get corrected is repeating the previous and current instructions by themselves. This can be improved by applying more various type of landmarks to the system.

5 Conclusion and Future Work

In this paper, we suggested a cost-effective indoor navigation aid system for people who are blind or visually impaired that uses no indoor navigation techniques. The user study shows that each step of the generated path instructions is clear enough for a user to reach the destination. Furthermore, the system gives shoreline-based instructions, such as by indicating walls and corner information, so that blind and visually impaired users can easily use the proposed system along with their traditional navigation aids, such as a white cane or a guide dog. The results show that the participants could follow the instructions easily, and the system could generate safe and effective paths for the user.

Future works will be related to the improvement of usefulness. First, the current system cannot generate paths and instructions in open spaces because the system is based on a wall in the building. Therefore, we will attempt to broaden the scope of the system using other landmarks, such as a chair, phone booth, or scent. This feature will be a good key for giving information about obstacles to users. In addition, we will include a recalibration feature to cope with situations where the user deviates from the desired paths.