Keywords

1 Introduction

Although human echolocation is a confirmed and relatively well-documented phenomenon [1, 2, 8, 9], the exact capabilities of average sighted or blind persons, and methods of teaching it to new students are not yet well studied [5, 6, 15]. A number of nearly “superhuman” echolocators are well known in the blind community [12] and sometimes offer training courses [4], which mainly consist of “tough love” training regimens in real-world environments that force students to utilize their sense of hearing [3]. The authors intended to prepare a less dangerous and demanding training regimen that could be disguised as a mobile phone game. Before development of the actual game the project began with a short study of echolocation itself and the possible binaural recordings that could be utilized in the developed software. The authors plan to repeat the tests in the future to observe improvement (if any) of the real-world echolocation skills after a training regimen with the binaural recordings is developed.

2 Experiments in Real Environments

As initial research on echolocation and as a benchmark for future testing of the skills of project participants, tests were prepared with real obstacles and three groups of participants: 10 blind children, 10 blind adults and 10 normally sighted adults. The tests were performed in three variations: indoors, in a padded and non-padded room, and outdoors.

2.1 Experimental Setup

The indoor tests were divided into 5 stages, each with 3 parts 9 questions each. In the first stage the participants localized a large 2 m × 1.9 m wooden wall, in the first part just its direction (left 45°, center or right 45°) at 1 m distance, in the second its distance (1 m, 2 m or 3 m) when it was in the center, and in the third part both the direction and distance had to be guessed. The second, third and fourth stage were analogous, but different types of obstacles were used: a 1 m × 1 m box, a 1 m wide × 1.9 m tall wall, and a 10 cm wide × 2 m tall cylindrical column. The fifth stage consisted of identifying the type and direction to an obstacle at a 1 m distance. An example task is shown in Fig. 1 along with the researcher’s position during the study. The outdoor tests only repeated the first two stages with 1 m and 2 m walls.

Fig. 1.
figure 1

Localization of obstacles by echolocation in a padded room using a mechanical clicker.

Full size image

The test participants were equipped with a dog-clicker to generate the echoes in a repeatable manner. They were asked to click once then answer, then click N times (up to 10) and answer again. Each time they answered they also judged their certainty in their answer on a scale from 1 (“I’m guessing at random”) to 5 (“I feel completely certain”). After providing the answer the second time, the testers were told if they were correct and if not, the correct answer was indicated to them by tapping of the obstacle. While the obstacles were moved, the testers listened to loud music over headphones. All sighted or partially sighted participants were blindfolded.

The results can be analyzed two ways: either to judge the echolocation skills of each individual or a given subgroup, or to judge the general difficulty of the various echolocation tasks. When comparing the tasks, the ones that could be considered “easiest” consisted of determining the direction to an obstacle, especially at the close 1 m distance. The tasks that were most difficult, answered at random or near-random level were the determination of the distance to an obstacle other than the 2 m wide wall, and determination of type of obstacle. Sample results are presented in Table 1.

Table 1. Averaged correctness of distance judgements in indoor trials.
Full size table

Comparing whole groups of users one can make several observations (Figs. 2 and 3). E.g. on average sighted persons performed worse than blind adults, especially expert echolocators [16], but better than blind children, which is consistent with data in literature [7, 11]. The blind adults fared better in the padded room, although the average difference in the percent of correct answers was minor. Bind adults were also very clearly more certain of their answers and correctly so. For all adults the certainty of an answer was good predictor of its correctness; however, for the blind underage participants, the correlation was not there (often children were very sure of their answers and surprised they were wrong).

Fig. 2.
figure 2

Comparison of correct results distribution for the different testing groups. The plot shows how many 9-question test set fell into each percent range (0–11%, 12–22% etc.) Each participant went through 6 such series for two sizes of walls. The results are compared to an expected distribution if all answers were random with 33% chance of correctness.

Full size image
Fig. 3.
figure 3

Averaged certainty vs correctness for all testers (10 blind adults, 10 sighted adults and 10 blind children) in the first two series of indoor echolocation tests (1 m wide and 2 m wide wall).

Full size image

The gathered results will serve as a benchmark for the project. The same groups of testers will practice echolocation tasks with various mobile applications and repeat the real scene tests afterwards, hopefully, demonstrating improved echolocation skills.

3 Experiments with Binaural Recordings

A Binaural Enthusiast BE-1 dummy head was used in the recordings with a torso constructed from a wooden skeleton and polyethylene foam. The mannequin was placed in the same position as the participants of the previously described experiments, with the head at 150 cm.

3.1 Clicker Sounds

The real-world echolocation trials used a dog-training clicker. The same clicker was initially used in the binaural recordings, but it was difficult to make the clicks consistently repeatable and to not disturb the mannequin’s position while clicking next to it. Instead, a recording of the clicker in a small anechoic box was made and played back looped through a UE Roll Bluetooth speaker.

Experiments were also done using variations of the clicker recordings with peak frequencies from 1 to 4 kHz and utilizing synthesized clicker recordings based on mouth-clicks of expert echolocators [13, 14].

3.2 Recorded Scenarios

The binaural microphone was used to record the same set of setups as during the padded room and outdoor tests as shown in Fig. 4.

Fig. 4.
figure 4

Binaural recordings were made in the same setups as the live echolocation tests.

Full size image

Additionally, recordings were made indoors in the school and university buildings, specifically in long hallways and on stairwells, as well as outdoors around university buildings, trees and parking lots.

3.3 Mobile Testing Applications

To perform tests with the binaural recordings a set of mobile applications was prepared that could be ran on Android or iOS devices. There were several goals of the initially prepared applications – to elaborate an efficient user interface (UI), judge plausibility of binaural recording use, develop best recording practices and gather echolocation sounds that were most promising for initial training.

The testing application allowed to manage users, recorded tests and exported data by emailing csv files. All test instructions, questions and answers were read by the OS’s default text to speech software (usually Talk Back on Android devices and VoiceOver on iOS). The prototype application was so far tested on a group of 10 persons: 3 sighted and 7 blind (2 with residual color/light perception).

3.4 Results and Discussion

The testing with the binaural recordings was much less formal than the real-world echolocation tests. Our approach was to prepare a set of 10–20 recordings, add them as a testing set to the application, have the 10 volunteers (including the authors) perform each set at least twice, gather and analyze the results. As the number of testing sets grew (currently over 30 tests with a total of 350 recordings), feedback from users and the gathered results were used to modify further recordings and questions, e.g. a single click was replaced by multiple clicks, the real clicker was substituted with a recording of that same clicker, then the recording was downshifted in frequency. Finally, the clicker recoding was replaced with a synthesized click based on an expert echolocator’s tongue click [14].

The full results will be published in a later paper are still being processed and too long to present in an abstract, but a number of conclusions can be summarized. The original binaural recordings of the indoor rooms with a mechanical clicker were insufficient to correctly echolocate obstacles (average correctness of locating large obstacles was at 38%). However, recordings made with a synthesized clicker (53%) and those made outdoors (63%) were significantly better. Some sound processing (e.g. amplitude compression or resampling) was shown to improve the perception of the echoes (by about 4–8%), especially when determining direction to an obstacle.

Dynamic recordings made by walking with the binaural dummy or mixing several consecutive static recordings also showed much more potential. Testers showed significantly more adept at identifying an obstacle that was getting closer than one that was static or moving away.

Interesting observations can be made when comparing answers given after first playback of a sound and after repeated playbacks. The repeated listening to a recording seems to have little to no influence on the correctness of answers in static recordings with a 3% average improvement, (when the standard deviation was 12%), but a more significant influence on dynamic recordings with a 12% improvement.

Similarly to the real-world tests, the number of repeated playbacks was actually inversely proportional to correct answers, suggesting that the sounds were listened to more often when the answer was difficult to determine, but the additional playbacks were not helpful in providing the correct answer. A further, more in-depth analysis of correlations between user certainty, types of questions, number of clicks and correctness of answers is currently being performed.

4 Future Work

As this paper is prepared, the echolocation studies continue to expand. Currently, tests with outdoor echolocation were completed. Dynamic tests are also being conducted, with participants tasked with approaching obstacles or localizing them off-path.

The binaural recording tests are also being continued. We are testing the players’ ability to imagine and interpret motion (preliminary results show very good understanding of linear motion, especially when approaching obstacles, but very poor understanding of rotation in a virtual environment).

The presented research will be used for the development and evaluation of a mobile game for teaching echolocation, as e.g. [17]. The game will feature binaural recordings in various environments and require the player to determines echoes from buildings, corridors and obstacles to choose travel paths. The initially prepared concept assumes a multistage game starting with simple “quiz-like” episodes based on binaural test sets that have so far shown most correct results. As the user trains, the plot of the game will change to more open tasks, where the user will be able to move by swiping the touch interface and/or select prerecorded paths. The main goal of the game is to deliver different combinations of sound and echo recognition tasks joined by a game plot. Player competition as one of the best mobilizing factors will be taken into account, as plan to have users compare high scores or even compete in tournaments.

5 Summary

We compared echolocation skills in a real world environment for three groups: blind children, blind adults and sighted adults. The test encompassed detection of obstacles at ranges from 1 to 3 meters in a padded and empty room. We also compared the accuracy of recognition of the same scenes using binaural recordings and a mobile application, as well as other tasks such as determination of direction of a corridor or building, or motion around obstacles. The gathered data will be used in the development and evaluation of a training game using binaural recordings.