Keywords

1 Introduction

Three-dimensional (3D) Virtual Reality (VR) technique has been widely used in many applications, including training, education and entertainment systems. However, VR applications based on Head Mount Displays (HMD) often have faced problems of navigation in the virtual environment due to the unrealistic sensing and synchronizing characteristics. Usually, conventional interaction techniques [1] depend only on the orientation of person’s moving body or direction of viewing scenes. The difference in the walking and viewing direction results in an error in navigating trajectory in the virtual environment, which makes the VR system unrealistic. It is inevitable to separate viewing direction and gait direction while exploring virtual environment, especially in VR training systems because the replication of the real situation is very important. In this paper, we propose an integrated framework for solving the problem using IMU attached to each shank and built-in gyro and acceleration sensors in Android mobile devices. In addition, this research aims to provide lightweight software as well as a wearable hardware solution to implement the Virtual Reality Walk-In-Place(VR WIP) [3] interface for training applications.

2 Configuration of the Proposed Framework

The user interface is composed of an Android mobile device having built-in 3-axis orientation sensor and a set of IMU attached to each shank. The IMUs detect gait with walking speed and direction, while, senors in the Android mobile device take information about the viewing orientation of the user’s head movement (Fig. 1).

Fig. 1.
figure 1figure 1

An illustration for difference between walking and viewing direction of Avatar in the VR training application.

Full size image

The Android mobile device can provide 3D VR head-mounted display within a user’s field of view, enabling a user to view virtual objects in the user’s surroundings. Proposed system has applied to a VR training system of disaster rescue from fire in an underground subway station. Figure 2 shows the implemented configuration of the VR training system with the proposed Integrated Framework. VR Contents are played on the Android mobile device, which also provides Bluetooth wireless interface linking to IMUs. The Avatar’s motion is synchronized with user’s movement, and viewing scenery is synchronized to user’s viewing direction. The user needs to escape from the underground firing palace to save her/his life in a limited given time. The user needs to find the best way to escape during the training, where the exit signs among the objects in the user’s view indicate the rescue route.

Fig. 2.
figure 2figure 2

Implementation of the VR training system with proposed integrated framework

Full size image

3 Experimental Results and Discussions

We need to recognize both the speed and direction of user’s gait in order to synchronize with scene displayed in the HMD and motion of the Avatar in the virtual space. Figure 3 shows 3-axis coordinate system of IMUs attached at the shank.

Fig. 3.
figure 3figure 3

Coordinate system of IMUs attached at the shank

Full size image

Figures 4 and 5 show measured acceleration of x- and z-axis, and angular velocity of y-axis during walking-in-place actions of the user. We can find a repetitive pattern matching to the each step of gait in the measured data. Variation in y-axis acceleration is negligible because there is a negligible amount of movement in the y-axis direction during walking-in-place motions as shown in Fig. 3. The repetitive pattern in the measured data indicates the step-up, the highest reaching point of foot, and end point of each step. We analyzed these patterns and applied this to machine learning technique to recognize walking state [4–6]. By using learning data, our framework can detect gait speed in real-time.

Fig. 4.
figure 4figure 4

Measured acceleration changes during walking-in-place. (1 g = 9.8 m/sec2)

Full size image
Fig. 5.
figure 5figure 5

Detection of walking event from the measured angular velocity of y-axis

Full size image

The walking direction, yaw angle of the user’s leg, can be detected by integration of angular velocity of x-axis. In order to obtain a better estimation of the user’s orientation, we can consider tilt from the output of accelerometers [2, 7] (Fig. 6).

Fig. 6.
figure 6figure 6

Detection of walking direction using the measured angular velocity of x-axis

Full size image

The sensors on Android head-mounted display (HMD) can detect a viewing direction in real-time, enabling to synchronize the virtual surroundings seen through the HMD [8]. Figure 7 shows logged moving trajectory of a Avatar with viewing direction in the disaster rescue VR training system. The scaling factor was used for each user to optimize the performance of the training system. These results indicate that developed algorithm and frame work can be applicable to the VR training system.

Fig. 7.
figure 7figure 7

Trajectory moving with viewing direction in the disaster rescue VR training system

Full size image

4 Conclusion

We present an integrated framework for real-time sensing and synchronization of both user’s moving speed with direction and viewing direction in walking-in-place experience for virtual training applications. The proposed solution can provide a simplified technique for implementing ‘Walking in Virtual Reality’ without omnidirectional treadmill. Experimental results indicate that the algorithm developed for Real-time Sensing of Gait and Viewing Direction can be favorably applicable to the VR training system.