Keywords

1 Introduction

People with blindness must be able to cognize their environment using acoustic information through their auditory sense when they are walking or conducting daily activities. This skill, known as “auditory orientation”, includes sound localization and obstacle perception. Sound localization is the ability to identify a sound source location, such as a vehicle or pedestrian. Obstacle perception is the ability to detect a silent object, such as a wall or pole, using sound reflection and insulation. It is sometimes called “human echolocation” [1].

Training of auditory orientation is usually conducted for people with blindness as an one lesson in orientation and mobility (O&M) instruction. Such O&M instruction is usually conducted in a real environment; the trainee is expected to acquire auditory orientation capability by listening to ambient sounds experientially [2]. However, training in a real environment where actual vehicles are running is sometimes danger and stressful for novice trainees. Furthermore, the trainee must spend a long time to acquire auditory orientation using this training method because it is very difficult for the novice trainee to discern and listen to important sounds selectively from many other environmental noises. To reduce the risk and stress, and to shorten the period of training, a new training method in an ideal sound field reproduced by acoustical simulation is considered very effective.

Some studies of acoustic training technology have been conducted in Japan and overseas to solve these problems [3]. However, these studies have covered only a very small part of auditory orientation, called “sound localization”. The acoustic training systems developed from these studies are too expensive to introduce into actual training sessions and are not suitable for practical use. Therefore, there is a need for a practical auditory orientation training system for safe and efficient rehabilitation to encourage the visually impaired to participate in social activities.

2 Auditory Orientation Training System (AOTS)

In 2003, National Institute of Advanced Industrial Science and Technology (AIST) and National Rehabilitation Center for persons with Disabilities (NRCD) started to develop a training method that combined “sound localization” and “obstacle perception”, and in 2005, we developed an auditory orientation training system (AOTS).

2.1 Composition

AOTS comprises ten 3-D sound processors (RSS-10; Roland Corp.), ten sound recorders/players (AR-3000; Roland Corp.), two sound mixers (RFM-186; Roland Corp.), a magnetic 6DOF position and direction sensor (3SPACE Fastrak; Polhemus), headphones and an amplifier (SRS-4040; Stax Ltd.), and a computer (iBook G4; Apple Computer Inc.). Software was developed (REALbasic; REAL Software Inc.) to function on Apple Mac OS X. The 3-D sound processors and sound recorders/players are controlled through MIDI; the magnetic sensor is controlled through RS-232C (Figs. 1 and 2).

Fig. 1.
figure 1

Outlook of auditory orientation training system (AOTS). AOTS was developed in 2005 by AIST and NRCD.

Full size image
Fig. 2.
figure 2

Application window of auditory orientation training system (AOTS)

Full size image

2.2 User Interfaces for Trainee

A trainee can listen to sounds in the virtual training environment through headphones while changing the head direction. A trainee can also walk though the virtual training environment by moving their feet. The head and foot movements are measured by the magnetic 6DOF position and direction sensors.

2.3 Elements of Virtual Training Environment

The virtual training environment of AOTS can include elements of four kinds: sound sources, walls, roads, and landmarks. The sound source can represent the sound of a vehicle, pedestrian, store, etc., and move in a constant speed and direction. The wall is used for training of the obstacle perception, and gives rise to sound reflection and insulation. The road and landmarks do not influence the sound propagation, but they are very helpful in the design of the virtual training environments (Fig. 3).

Fig. 3.
figure 3

Elements of virtual training environment. (a) Sound, (b) Wall, (c) Road and (d) Mark. (e) Example of the virtual training environment.

Full size image

AOTS can reproduce six sound sources and four ambient noises (from east, west, north, and south), simultaneously. To reproduce the presence of a wall for obstacle perception training, AOTS can reproduce reflection and insulation of ambient noise, and insulation of moving sounds. These reproductions enable the trainee to learn to detect walls and paths. Reflection and insulation of the ambient noises are reproduced when the listener approaches to within 2 m of a wall; sound insulation is reproduced by attenuating the sound by 6 dB. The O&M instructor can design and “construct” the virtual training environments easily by describing them in extensible markup language (XML), which was originally proposed for this system.

2.4 Evaluations [4]

Some effectiveness assessments of AOTS were conducted. Subjects were 30 sighted people who had been blindfolded. They were divided into three groups: Control, AOTS, and O&M. The Control group was not trained. The AOTS group was trained using AOTS. The O&M group was trained using a usual O&M program. The training course was a 50-m-long straight sidewalk.

Results show that actual O&M training is effective for reducing stress, although novice trainees feel great stress initially. AOTS was also effective, but slightly less so than O&M (Fig. 4).

Fig. 4.
figure 4

Reduction effect of stress [4].

Full size image

The veering reduction effect of AOTS was also measured using a travel locus. Results show that AOTS is the most effective method for training auditory orientation skills. A possible reason is that no other factors (tactile, smell, etc.) were included in the virtual training space of AOTS. Therefore, the trainee was able to concentrate on learning the auditory orientation (Fig. 5).

Fig. 5.
figure 5

Reduction effect of veering [4]

Full size image

2.5 Ploblems

However, at about 5 million yen (about 50 thousand dollars) the system was expensive. Also, it was too large to carry and enabled the position and orientation of the head to be measured over a distance of no more than 1 m at a time. Thus trainees cannot walk while using it.

3 Wide-Range Auditory Orientation Training System (WR-AOTS)

Since 2008, Author has been jointly attempting to reduce the size of the auditory orientation training system with Tohoku University, et al., to expand its coverage, and to reduce its cost, and finally developed an improved system.

3.1 Composition

The developed training system consists of special-purpose software, “WR-AOTSTM (Wide-Range Auditory Orientation Training System)”, a personal computer (PC), a stereo headphone set, and a commercially available game controller (for wide-range positioning) (Figs. 6 and 7).

Fig. 6.
figure 6

Outlook of wide-range auditory orientation training system (WR-AOTS). WR-AOTS was developed in 2013 by AIST, Tohoku University, et al.

Full size image
Fig. 7.
figure 7

Application window of wide-range auditory orientation training system (WR-AOTS).

Full size image

3.2 Improvement

Three-dimensional sound processing to artificially reproduce the auditory orientation clues used by a visually impaired person during walking is achieved by calculation with a commonly available general-purpose PC central processing unit and “SifASoTM (Simulation environment for 3-D Acoustic Software)” technology of Tohoku University [5], without the need for an expensive dedicated DSP (digital signal processor). To measure head position, an inexpensive, low-precision wide-range positioning technology has been introduced. Its precision is stabilized by software processing, and it uses the built-in acceleration and gyro sensors in a commercially available game controller costing several thousand yen (several dozens dollars), instead of expensive, high-precision narrow-range positioning technology priced from several hundred thousand to several million yen. These improvements have resulted in a substantial reduction in the cost of auditory orientation training. The system has been made compact by using a laptop PC, allowing the trainee to walk with it. The trainee can receive training safely while walking in a spacious area free of obstacles, such as in the grounds of a blind school (Fig. 8).

Fig. 8.
figure 8

Principle of wide-range auditory orientation training system (WR-AOTS)

Full size image

3.3 Distribution

In April 2013, the first version of WR-AOTS Ver. 2.04 was released and the minor updated version Ver. 2.06 was started to be distributed for the blind rehabilitation and/or education facilities.

4 Updates of WR-AOTS

After the first version was released, we have been keeping providing update on the facilities’ demands.

4.1 Ver 2.07

Turning of Sound. The most remarkable request from the O&M instructors was to enable the sound source to turn a curve. The sound image motion when the car runs straight, turns a curve, and runs away is very important cue to detect a crossroad for the Blind people. In Ver. 2.06 and before, the Sound could move only straight in constant velocity. In the updated version Ver. 2.07 and after, the Sound can turn. The situation where a car turns a curve can be reproduced by this improvement. The training to detect a crossroad by using car running sound can be performed. The blind people can learn how to detect a crossroad by sound motion in our system (Fig. 9).

Fig. 9.
figure 9

Turning of sound. motions of deceleration, turning, and acceleration are calculated automatically.

Full size image

Volume Control of Sound. In Ver. 2.06 and before, volume of the Sound could not be controlled. In Ver. 2.07 and after, the volume can be controlled. It is possible that specific Sound is loud and other Sounds are small. The difficulty of the listening training can be adjusted by controlling volumes.

Kind, Direction, and Volume Control of Ambient Sounds. In Ver. 2.06 and before, kind, direction, and volume of the ambient sounds could not be changed. In Ver. 2.07 and after, these can be controlled.

  • For the kind of the Ambient sound, constant white noise (that includes all frequency component) and pink noise (lower frequency is stronger) can be selected as well as conventional town noise. They are suitable for the novice trainee, who feels difficulty to use complex town noise, to learn how to listen to the specific sound in the ambient noises, or acquire the obstacle perception by using reflection and/or insulation of the ambient sounds.

  • For the direction of the Ambient sound, as well as the four points of the compass, two directions or one direction can also be selected. Generally, wall can be detected easier as less direction. Thus, they are suitable for the novice trainee.

  • The volume of the Ambient sound can be controlled. The difficulty of the training for listening to specific sound in ambient sounds can be adjusted.

4.2 Ver. 2.08

Screen Reader. In Ver. 2.07 and before, screen readers could not be used. In Ver. 2.08 and after, principal buttons etc. can be read by screen readers, and can be selected by TAB key. Operation is available by screen reader and keyboard in some degree.

Training Field Data Based on Japanese Rules of Public Sounds. In Ver. 2.08 and after, the sounds of the accessible pedestrian signals (APS) and the silent vehicle approach notification equipment are added. Japanese rules of public sounds that must be learned in O&M can be learned by this system.

5 Conclusions

Authors started to develop a training method that combined “sound localization” and “obstacle perception” by using acoustic virtual reality technologies for the orientation and mobility training for the blind people in 2003, and we finally developed an auditory orientation training system (AOTS) in 2005. As a modified version of AOTS, the first WR-AOTS was released April 2013 for the blind rehabilitation and/or education facilities. By January 2015, about 70 requests for use of it were received from the blind rehabilitation and/or education facilities. We will keep providing update on the facilities’ demands in future.