Keywords

1 Introduction

Augmented Reality (AR) refers to the combination of virtual objects with the real world (Azuma, 1997). Recently, many researchers have developed applications for adapting AR into academic and industrial settings (Lee, 2012). The AR environment can improve individuals’ performance significantly by reinforcing their perception and improving their contact with the real world. In this study, we investigated the learning effect of the AR training about an anti-air warfare coordinator (AAWC) task. The time window-based human-in-the-loop (HITL) simulation was used as a tool to measure participants’ task performance during the experiment (See Fig. 1). This HITL simulation is a radar monitoring simulation. An operator must defend his/her ship against hostile aircraft (Kim, Rothrock, Tharanathan, & Thiruvengada, 2011; Macht, Nembhard, Kim, & Rothrock, 2014). Several Rules are embedded in the simulation so that participants must learn how to execute these task-specific rules during the training exercises.

Fig. 1.
figure 1figure 1

Anti-Air warfare coordination human-in-the-loop training simulation

Full size image

To conduct the experiment, the research team developed two different training methods: AR training system and traditional training textbook. Participants in Group A were trained on how to perform the AAWC task using the AR training, and Group B was taught with the traditional training textbook. During the experiment, the AR training system portrayed the AAWC training contents in 3D. This created a more exciting way to learn the information. To develop a collaborative AR environment and appropriate interactions between human and the AR system, the Oculus Rift goggle and Leap motion controller were used. The goggle allows participants to see the training material in 3D, such as demonstrating airplanes in 3D that fly in time with the different warning rule conditions. The motion controller provides learners an easy way to navigate the AR training system.

2 Literature Review

Many AR research studies support positive training effects of using the AR environment. First, the AR environment enables people to gain spatial relationships among complex ideas (Arvanitis et al., 2009), because the AR increases people’s spatial skills (Martín-Gutiérrez et al., 2010). In addition, the AR can help learners to experience abnormal events that do not naturally occur in real life (Kaufmann, Steinbügl, Dünser, & Glück, 2005). By using 3D virtual objects, it is possible to create the abnormal events during the training session. This realistic interactive learning experience can reinforce learners’ cognitive mapping regarding those events. Therefore, currently, the AR combines virtual objects or data with physical objects to create environments that allow one to think of invisible actions or ideas (Wu, Lee, Chang, & Liang, 2013). The most powerful advantage for using AR is improving performers’ enthusiasm and attention and increasing their analytical skills (Wu et al., 2013). For that reason, this technology can create a novel experience, which decreases people’s boredom and increases understanding during learning. (Lee, 2012, Li, 2005).

The AR environment is not only good for academic domains, but also for other industries such as medical (Kamphuis, Barsom, Schijven, & Christoph, 2014) and air traffic control environment (Hofmann, König, Bruder, & Bergner, 2012). For example, AR is used as an aid in making surgeries productive. Most surgeons normally develop mental pictures of where the surgery needs to take place. For that reason, they developed a method to find where the surgery must be performed. Shuhaiber (2004) also found that the AR of overlapping the images on a live video camera can assist surgeons to develop the desired mental pictures of the surgery.

3 Method

3.1 Participants

In this study, we have led to research efforts in AR training of AAWC task. 24 undergraduate students in the Junior and Senior levels participated in the experiment. During the training session, 12 students completed the training lesson by using the AR training (Group A), and the other 12 students completed the training lesson using the AAWC training textbook. Participants studied from one topic to the next one at their own speed. This experiment did not restrict anyone based on gender, ethnicity, or religion. Students, however, who had similar previous experience were excluded from this experiment.

3.2 Measures

To measure the performance of AAWC task, Situational Awareness Global Assessment Technique (SAGAT) was used. SAGAT is designed for the real time human-in-the-loop simulation such as an aviation monitoring or military cockpit (Endsley, 1988). This technique was used to collect objective data of SA across all the participants. Participants answered SA questionnaires after the simulation was stopped at random times. The responses were compared to the correct answers in the computer database. SA is defined as the awareness of the environment within time and space (Endsley 2012). The accuracy of situation awareness (SA Accuracy) is calculated by:

$$ SA\,accuracy = \frac{Number\,of\,correct\,response}{Total\,number\,of\,SA\,probes} \times 100 $$
(1)

3.3 AAWC Human-in-the-Loop Simulation

In this experiment, participants were trained on how to use the AAWC Human-In-The-Loop simulation. The participants took on the role of anti-air warfare coordinator. They were assigned to identify unknown aircraft and to apply rules of engagement (ROE). Participants trained to take the appropriate actions on the unknown or hostile aircraft. In the AAWC simulation, the participants must focus on the radar screen to find unknown aircraft as soon as possible. They also need to perform the ROE as accurately as possible.

The ROE consists of three main tasks (see Fig. 2): identification task (Plan 1), warning task (Plan 2), and assign task (Plan 3). Figures 3, 4, and 5 show the hierarchical task analysis (HTA) charts for these respectively.

Fig. 2.
figure 2figure 2

Rules of engagements

Full size image
Fig. 3.
figure 3figure 3

HTA chart for identification task

Full size image
Fig. 4.
figure 4figure 4

HTA chart for warning task

Full size image
Fig. 5.
figure 5figure 5

HTA chart for assign task

Full size image

Identification Task. The main goal of this task is to identify the unknown aircraft as soon as possible. There are two types of identification: (1) Primary ID, (2) Air Type ID (see Fig. 3).

If the participants decide to update Primary ID, they must go through Plan 1.1. First, the participants click “Primary ID” button. Then, they will select ID. If a participant chooses the unknown aircraft is a friendly aircraft, then he or she should click the “friend” button. After that, the selected aircraft will change its status from unknown to friendly aircraft.

If the participant would like to update Air type ID, he or she should follow Plan 1.2. To perform the Plan 1.2, click “Air Type ID” button. Then, select one of the Air IDs: Missile, DCA, Strike, AEW, Non-military, Helicopter, or Clutter.

Warning Task. If the participants would like to complete the warning task, they should follow Plan 2 (see Fig. 4).

In order to complete Plan 2, participants should click on the “I.D.S.” button. After that, they should choose the warning level. If the unknown or hostile aircraft is 50–40 nautical miles (NM) away from the own ship, then they should click on the “Issue level 1 Warning” button. If the aircraft is 40–30 NM away from the own ship then click on the “Issue Level 2 Warning” button. If the aircraft is less than 30NM from the own ship, then click on the “Issue Level 3 Warning” button.

Assign Task. If the participants would like to complete the assign task, then they should follow the Plan 3 (see Fig. 5). To perform the Plan 3, the participants should execute plan 3.1, 3.2, 3.3, 3.4, and 3.5 in sequence.

3.4 Augmented Reality Training System

For the AR training, Oculus goggle and Leap Motion were used to create the AR environment (see Fig. 6). The Oculus goggle provides 3D virtual images with the real view. The Leap motion recognizes the participant’s swiping gesture to turn the AR training slides. The AR training consists of 6 lessons: (1) introduction, (2) how to control the radar display, (3) how to acquire proper information, (4) how to perform the identification task, (5) how to perform the warning task, and (6) how to perform the assign task. After each lesson, the participants will have a chance to practice what they have learned through the AR training slides.

Fig. 6.
figure 6figure 6

Experimental setup for AR environment

Full size image

3.5 Procedure

The total experimental time for each participant was about 4.5 h over a course of 3 days. Every participant executed 6 scenarios. Each scenario has 10 aircraft: 6 unknown, 3 friendly and 1 defense counter aircraft (DCA). The participants controlled the DCA to identify the unknown aircraft ID. The DCA helps them to collect a clear data about the aircraft identify. On Day 1, every participant learned how to use the AAWC simulation. However, Group A and B used different training methods. Group A learned by using the AR Training Method. Group B participants learned the information using the traditional training textbook. On Day 2, participants completed 3 scenarios. After each trial scenario, the participants asked to answer 9 SA questions. On Day 3, they completed another 3 scenarios. After each trial, they also asked answer 9 questions. So, each participant must answer 54 SA questions for two days.

4 Results

The ANOVA result shows there was a significant performance difference between Group A (M = 58.02, SD = 22.40) and Group B (M = 46.42, SD = 16.95), F (1,12) = 12.29,p < 0.01. The Group A’s SA accuracy was higher than Group B’s (see Fig. 7). In addition, the learning curves in Fig. 8 support that the AR training can give a positive learning effect in the computer-based military training simulation.

Fig. 7.
figure 7figure 7

Interval plot of SA accuracy between group A and B

Full size image
Fig. 8.
figure 8figure 8

Line plot for SA accuracy by trials

Full size image

5 Discussion and Conclusion

In this study, we found the performers who were trained by the AR training showed higher SA scores compared to those who used the textbook. One explanation of this is the AR environment provides the participant an improved connection between declarative knowledge of AAWC rules and cognitive mapping of ROE. According to the research done by Kaufmann (2002), viewing objects in 3D and having the ability to interact with those objects can improve a learner’s understanding of three-dimensional geometry. Another study also shows the students could effortlessly view not only the desired catalyst, but also its spatial structure when it reacts with another molecule (Maier, Tönnis, & Klinker, 2009). In the experiment, the radar monitoring screen displays all tracks in a 2D environment (see Fig. 1). However, the flight path of an aircraft is in a 3D environment. To be aware the accurate flying pattern of all aircraft, the learners need to develop an ability to draw the detailed 3D map of space in their heads. By using the AR training, participants learn effectively, how to interpret the 2D information as the 3D information.

Another explanation is the AR environment stimulates the motivation to learn about AAWC rules. The Group B less paid attention to the materials being taught compared to the participants in Group A, because the AR contents created an interactive environment to boost participant’s interest in learning. Many research studies have found a similar positive effect on learner’s motivation because of AR (Di Serio, Ibáñez, & Kloos, 2013; Liu, Tan, & Chu, 2007; Medicherla, Chang, & Morreale, 2010).

In this study, we have found the AR training method is better than the traditional textbook method to improve learners’ situation awareness during the AAWC training. This result tells us the AR environment is helpful to train developing a cognitive map of 3D space. Moreover, the participants expressed higher interest in the AR training through learning. One limitation of this study is the each participant had only experienced 1.5-hour a day training and 6 trials for the HITL simulation. In order to understand the long-term learning effect of AR training, the experiment should last longer.