Keywords

1 Introduction

Head-mounted displays (HMDs) offer virtual reality (VR) gaming that exploits visual and auditory senses. For example, “The VOID” [1] is a large-scale VR attraction that enables players to play games while walking around a wide area. However, using HMDs can lead to problems including VR sickness or discomfort due to compression of the head, and HMDs are often not sanitary due to sweat when used by large numbers of people. Also, children under the age of 13 are recommended not to use HMDs as wearing them may cause health problems.

VR devices utilizing senses other than visual and auditory senses have also been developed. Nakagaki’s Linked-Stick [2] is a stick-based device that can mimic the motion of someone else using a stick-like tool; for example, a conductor, a baseball batter, and a swordsman. Striker VR [3] is a gun-type device that can recreate the feeling of shooting a gun. Electric bow interfaces [4, 5] use a real Japanese bow with the installation projector or real bows with the mobile laser projector. Moreover, there is also a pregnancy experience system that enables a user to affect the virtual fetus’s behavior with gentle or violent movements [6].

We have developed VAIR Field, a mobile VR system that enables multiple players to play a battle game with smartphones. As we prioritize user-friendliness for children, VAIR Field uses smartphones not HMDs to display images since smartphones are lightweight and easy to handle. This allows players to look around and avoid obstacles while handling their device, so they can safely move around without hesitation while they are playing. However, VAIR Field cannot superimpose images of guns and swords on the controller or replace walls and obstacles in rooms with forests or outer space.

Considering these things, we developed VR shooting games that involve using infrared guns, BB bullets, and paint bullets for VAIR Field. In VAIR Field, gun-like and bow-like devices can be used that combine a smartphone, a positional tracking sensor, and a server integrating them. By using VAIR Field, we can implement VR or augmented reality (AR) shooting games with multiple players without HMDs as shown in Fig. 1.

In this paper, we describe VAIR Field and its devices: a new VAIR Gun and VAIR Bow. We also compare the new VAIR Gun and VAIR Bow with the initial VAIR Gun and our previous electric bow interface.

Fig. 1.
figure 1

Depiction of playing VAIR Field

Full size image

2 VAIR Field

2.1 Concept

VAIR Field is based on three concepts. The first is user-friendliness for children. Thus, it does not use a HMD, which can cause health problems and is often unhygienic. Also, children tend to treat devices roughly, so the device must be robust and difficult to break. Moreover, it must be sufficiently safe to use.

The second concept is sports-like play. VAIR Field is not just a technical demonstration but also needs to offer high quality in terms of appearance, accuracy, response speed, and handleability. Especially, players can be expected to improve their shooting ability by physically handling the VAIR Gun and VAIR Bow.

Finally, the third concept is “feel of things.” This means something that physical objects possess the right temperature, feel, hardness, weight, ease of use, and affordance. The VAIR Gun and Bow require sufficient affordance to enable players to understand how to immediately handle them. Of course, interactions such as recoil and reaction when using VAIR devices are also important. For example, the VAIR Gun should be shaped so that the player can understand immediately how to grasp and shoot without instructions and should feel stable, have the right weight balance and hardness, and be comfortable to touch. Also, the VAIR Gun should produce recoil and sound after shooting.

Fig. 2.
figure 2

System flow of VAIR Field

Full size image

2.2 System

Figure 2 shows the system flow of VAIR Field. VAIR Field consists of a server combining the two VAIR devices (VAIR Gun and VAIR Bow), a HTC Vive tracker, and a Windows Server. Each device consists of the HTC Vive tracker and a smartphone. We use the HTC Vive tracker, which is the best positional tracking sensor, to make VAIR Field sufficiently scalable that multiple players can play. This tracker cannot be used without HTC Vive (which is a HMD), so VAIR Field needs to connect to Windows to access HTC Vive.

Figure 3 shows a minimum floor plan for playing VAIR Field. Multiple players can play with the VAIR devices in any room meeting the requirements of this minimum plan.

Fig. 3.
figure 3

Minimum floor plan for using VAIR Field

Full size image

2.3 Registering Obstacles

As shown in Fig. 3, a player’s has more potential tactics when there are obstacles and walls than when playing in an open space. In fact, such obstacles are also used in competitions such as games using infrared guns and bows with archery tabs. VAIR Field also utilizes obstacles. Although VAIR Field cannot yet perfectly recognize and synchronize with obstacles, it can register existing obstacles in advance.

Fig. 4.
figure 4

Positional synchronization of virtual and real objects

Full size image

For example, by inputting data of a real object’s position and scale to the VAIR Field, the object can be used as an obstacle in VR space (Fig. 4). When a player hides behind the object, he or she can hide behind the obstacle in VR space and can play in a shooting fight using realistic tactics. Since the real space is clearly visible in the mobile VR, a player can safely run around, which is impossible with a HMD.

Therefore, VAIR Field enables players to move in a sports-like way much more freely than VR using a HMD.

Fig. 5.
figure 5

Screen in play. (The blue pin in the dashed red circle is another player.) (Color figure online)

Full size image

2.4 Displaying Other Players

Figure 5 shows a screen displayed on a smartphone while playing VAIR Field. The opposing player is displayed as a blue pin in the dashed red circle. We measured the distance to the screen from a player’s eyes when the player stretches his arm forward. We calculated the viewing angle so as to display the scale correctly. However, since the screen of the smartphone is small, the viewing angle becomes too narrow, and it is difficult for the player to understand what is happening in the surrounding area. Therefore, we set the viewing angle slightly wider, so it is hard to say whether the screen displays images that are sufficiently realistic.

3 VAIR Device

3.1 VAIR Gun (Old)

Figure 6 shows the initial VAIR Gun, which uses a HTC Vive controller for tracking its position. Also, its trigger is the controller’s trigger. This gun uses the controller, so its system flow is slightly different from that of the current VAIR Gun.

Fig. 6.
figure 6

Initial VAIR Gun

Full size image

First, the initial VAIR Gun uses the controller’s positional tracking sensor and trigger information. After being sent via Bluetooth, the information is input to VAIR Field software in Windows via the HTC Vive headset. This software, built by Unity, uses the server client model of network games and Windows functions as a server. Each smartphone’s client software, also built by Unity, connects to the server via Wi-Fi and then performs pairing with the controller. Next, by using the pairing information, the server sends information received from the corresponding controller and game information to the smartphones via Wi-Fi. Finally, by using their information, rendering is done on each smartphone.

This framework is scraped out of aluminum alloy (a6061) for physical robustness, it is sandblasted for a satin finish, and the grip part is painted lacquer on walnut. The upper parts are molded with a 3D printer and painted to look like metal, and the muzzle that fixes the parts is a screw made of aluminum alloy (a5052).

Using these components, this initial gun is designed to imitate a real gun. However, some parents who saw and tested the gun at the Taipei Game Show 2017 claimed that they did not want their children to have it because this device looked too realistic. Therefore, this initial gun is not currently used for VAIR Field.

Fig. 7.
figure 7

Current VAIR Gun. (The center is a model for iPhone.) (Color figure online)

Full size image

3.2 VAIR Gun (Current)

Figure 7 shows the current types of VAIR Gun, which have a science fiction-like appearance and do not look like real guns. However, their forms enable players to recognize easily that they are gun-like devices. These guns use the HTC Vive tracker as a positional tracking sensor, so their trigger functions are independent. Because their magazines and main bodies are separated, players must exchange the magazines when reloading. Hence, if the magazines are not loaded, the guns cannot shoot bullets.

As shown in Fig. 7, there are three types of magazines: red, silver, and black. The magazines have different numbers of bullets with different power and automatically change the viewing angle when loaded. For example, the red magazine contains few bullets with high power, like ammunition for a sniper rifle. Also, when the red magazine is loaded, the viewing angle is automatically zoomed in, enabling the player to shoot more precisely, like a sniper.

Fig. 8.
figure 8

Current VAIR Gun and magazines. (Press the lever on the left rear side to release the magazine. By pushing in a magazine from behind, it is automatically locked and loaded with bullets.)

Full size image

These magazines are loaded and released by imitating the exchange mechanism of a real gun’s magazine. As shown in Fig. 8, there is a release lever on the left rear part of the device, and by pushing this, the magazine lock is released and the magazine can be pulled out. The magazine can be pushed in and pulled out from the back of the device, locking is automatically applied by pushing in the magazine, and bullets are loaded.

The magazine (i.e., Mag A in Fig. 2) contains a battery, an actuator, and a control board. In other words, this part is the main body of the VAIR Gun. To identify each magazine, its ID is electronically set in the control board. The magazine contains a pogo pin and a pogo pin receiver. When the magazine is loaded, the pogo pin and its receiver are connected. After connecting them, the magazine’s ID information is sent to the server of the Vive tracker via Bluetooth. At the same time, the trigger is connected to the magazine. When the player pulls the trigger, the information is transmitted to the inside of the magazine, the actuator is activated immediately upon receiving the trigger, and a reaction that varies depending on magazines is generated. Then, this information is sent to the Vive tracker, which then sends the same information as well as information of position and angle to the Windows PC via Bluetooth. After this processing, this current gun operates in the same way as the initial gun.

Fig. 9.
figure 9

Fully equipped VAIR Gun. (Spare magazines can be stocked inside the cowl.)

Full size image

Most parts of the VAIR Gun were produced by using 3D printers, so there are few metal parts. Compared with the initial gun in Fig. 6, the new gun has about three times as many parts but is 100 g lighter. It also has a standard Picatinny rail and can be equipped with not only an iron site but also a commercially available dot site. Moreover, it can be customized, such as by attaching a cowl (Fig. 9). The parts that hold a smartphone can be easily reattached, so players can attach any smartphone such as an iPhone, iPod Touch, and Android.

3.3 VAIR Bow

The VAIR Bow (Fig. 10) is a bow-like device that has a similar basic structure to the VAIR Gun. It consists of the rims, strings, dampers, and stabilizers of a real bow. It uses the Vive tracker as a positional sensor and can also attach any smartphone. The best feature is that the shooting intensity depends on the amount the string is pulled.

Fig. 10.
figure 10

VAIR Bow and depiction in use

Full size image

The grip consists of an aluminum alloy (a 6061) sandwiched between wood. The upper uses parts output by a 3D printer. Four strain gauges are affixed on the aluminum alloy just under the upper rim. They constitute a Wheatstone bridge circuit and measure distortion of aluminum. It can calculate the amount of force applied when a player pulls the string. The value of the distortion is passed through the amplification circuit and is sent to the Arduino Fio. Then, by using Xbee, the information is sent to a Windows PC via Wi-Fi. However, since a standard Windows PC cannot receive Xbee, it connects to a USB Xbee receiver.

The VAIR Bow uses the Vive tracker as a positional sensor and the Arduino Fio for the input information separately, so its structure is complicated. Although the bow weighs 4.5 kg, it is still not lightweight enough for children under 13. Therefore, VAIR Bow needs to be lightened and its structure improved.

4 Comparison

Table 1 shows specifications of four devices using the Vive controller or tracker and our previously developed 3rd Electric Bow. Compared with a general-purpose Vive controller, the initial VAIR Gun (Old) using it has basically the same performance and is heavier because of the added exterior. For the initial and current VAIR Guns, the strength of the trigger cannot be adjusted by changing the tracking sensor from the Vive controller to the Vive tracker. However, the current VAIR Gun is built so that trigger strength does not need to be changed.

Table 1. Comparison of specifications of devices using Vive controller or tracker, and previous work. (* means the weight without a smartphone.)
Full size table

On VAIR Field, up to two controllers and 15 trackers can be used with wireless per a set of Vive. Of course, if there are multiple sets, the numbers of controllers and trackers increase accordingly. The current VAIR Gun is about one-third lighter than the initial one because it mainly uses resin parts. Moreover, when using the current VAIR Gun, a player can exchange magazines and handle it like a real gun because the player can aim at a target with a real site.

Furthermore, the current VAIR Gun has three types of magazines. The red and silver magazines that have the recoil mechanism weigh 106.3 g and 84.5g, and the black magazine that does not weighs 14.6 g. The weight of the current VAIR Gun (357.1 g) in Table 1 includes the medium magazine weight of 84.5 g. Without this magazine, the VAIR Gun weighs 272.6 g, about half as much as the initial VAIR Gun (533.6 g). When playing, the weight of the smartphone is added.

Comparing the bows, the VAIR Bow can track positional information better than the 3rd Electric Bow [5], which can detect only rotation. For displaying images, all VAIR devices use a smartphone whereas the 3rd Electric Bow uses a mobile laser projector. The 3rd Electric Bow displays images on the wall, so its screen is larger than that of a smartphone. Since the 3rd Electric Bow’s viewing angle of projection is not very wide, the player does not feel the difference in screen size between images.

The 3rd Electric Bow projects images onto a wall, so multiple people cannot use it in the same space at the same time. It contains a PC, a projector, and a battery, so it can work as a standalone device. However, it is so heavy (about 1697g) that a player has difficulty using it for a long time. On the other hand, the VAIR Bow weighs less than 1 kg without a smartphone. When an iPhone 7 is attached to the VAIR Bow, it weighs 1057.5 g. The VAIR Bow is about one-third lighter than he 3rd Electric Bow, so it is comparatively lightweight.

5 Conclusion

From the comparison of specifications, we conclude that the new VAIR Gun and Bow mostly realize VAIR Field’s concepts of user-friendliness for children, sports-like play, and “feel of things.” However, the space in which Vive can be used is too narrow for multiple people, so tracking in a wider space needs to be achieved. In the future, we plan to enable more people to use VAIR Field at the same time.