Keywords

1 Introduction

According to the education literature, the concept of Edutainment refers to creating a learning activity that fosters excitement, engagement, and enjoyment [1, 2]. A form of Edutainment that encourages learning through the use of video games is Game-based Learning. Video games are web- or PC-based systems used often for entertainment purposes, but have applications that extend to a multitude of domains such as education, therapy, simulation, etc. The term “serious games” has evolved from Game-based Learning to describe video games for entertainment that promote learning and contain elements of instruction [3, 4]. The two instructional gaming elements of interest include excessive positive feedback and competition. Excessive positive feedback elicits a sense of power and control in the learner by providing the individual with specific information regarding their performance [5, 6]. In order for this strategy to be effective, it must not impede the learner’s task objectives. On the other hand, competition provides the learner with an increasing desire to win. By utilizing a leaderboard, the individual is motivated to do well in the task which ultimately leads to learning. These instructional elements can be applied to Simulation-Based Training (SBT), a method for providing structured practice and learning experiences. One area of application for SBT is the Military domain. Within the Military domain, there are increasing efforts to incorporate SBT as a supplement for classroom-based instruction and live-training on Behavior Cue Analysis. Behavior Cue Analysis is a decision-making procedure focused on training that provides the necessary preparation for strengthening Warfighters’ perceptual skills. Warfighters are trained to proactively identify anomalies in an irregular and ambiguous environment by means of six domains of behavior cues (i.e., Biometrics, Kinesics, Proxemics, Geographics, Atmospherics, and Iconography/Symbolism) [7]. The procedure of Behavior Cue Analysis involves determining a baseline, detecting and classifying behavior cues, and deciding whether the behavior is an anomaly in the baseline. Figure 1 illustrate the process of Behavior Cue Analysis.

Fig. 1.
figure 1

Behavior cue detection [8]

Full size image

This research effort focuses on investigating personal cues (i.e., the behaviors of an individual), specifically cues in the Kinesics domain. Kinesics is the manner in which nonverbal messages are conveyed [9, 10]. Some examples of kinesic cues are gestures, facial expressions, body language, and body positioning. Past research has examined the effectiveness of kinesic cues for Behavior Cue Analysis training using SBT [11, 12]. For this research effort we examined four kinesic cues (i.e., check six, clenched fist, slap hands, and wring hands). Table 1 provides a description and classification for each cue.

Table 1. Kinesic cue descriptions and classifications
Full size table

This research augments previous efforts on the Kinesics domain by investigating instructional gaming strategies that elicit a motivation to learn. The three conditions (i.e., control, excessive positive feedback, and competition) were assessed using performance measures (e.g., detection and classification accuracy). The purpose of this study compared different Game-based Training (GBT) conditions to assess their effectiveness for Behavior Cue Analysis training. These findings can provide policy makers with empirical evidence for understanding the use of GBT as a method of instruction. Furthermore, the research and training communities can benefit from this line of research by utilizing GBT strategies for detecting Kinesic cues using a VE.

2 Method

2.1 Participants

A total of 91 participants were recruited from the University of Central Florida and the surrounding community. The sample included 44 females and 47 males, between the ages of 18 and 40 (M = 21.47, SD = 3.71). All participants were U.S. citizens and required to have normal or corrected to normal vision. To adhere to the U.S. Army vision requirements, participation in the experiment required full color vision according to the Ishihara’s Tests for Colour Deficiency [13]. Previous participation in Behavior cue detection studies was restricted due to the similarities between the experimental tasks. Participants were compensated $10 per hour for a duration of 1.5 h of participation.

2.2 Materials

Virtual Battlespace 2 (VBS2) Version 2.0 development software was used as the platform for this study, because of the flexibility to customize scenarios and its high quality display of kinesic cues.

2.3 Experimental Design

A between-subjects design was conducted using one independent variable (i.e., feedback) with two levels (i.e., excessive positive feedback and competition) and two dependent variables (i.e., detection accuracy and classification accuracy). A control condition was also added to compare the two levels of feedback and asses performance. For the control group, participants were asked to detect and classify targets as exhibiting aggressiveness or nervousness without any instructional support. In contrast, during the excessive positive feedback condition a small, green box appeared after every correct identification, stating “Correct Cue Identified Good Job-Keep Going!” For every three consecutive positive identifications, a larger notification box appeared stating “Three positive Identifications in a row Hit Streak Good Job, Keep Going!” Figure 2 displays the two forms of feedback. Thirdly, in the competition condition, participants were informed that a leaderboard (see Fig. 3) will be shown at the beginning and the end of the experiment, as well as after each vignette. The leaderboard will show the participant’s performance score. Finally, Detection accuracy measured the ratio between number of correctly identified kinesic cue targets and the total number of kinesic cue targets. Classification accuracy measured the ratio between the number of detected and classified kinesic cue targets and the total number of kinesic cue targets.

Fig. 2.
figure 2

Representation of the excessive positive feedback condition

Full size image
Fig. 3.
figure 3

Representation of the leaderboard in the competition condition

Full size image

2.4 Procedure

Upon arrival, participants were asked a series of pre-experimental questions, in addition to taking the Ishihara Test for Colour Blindness [13], to verify that he or she fulfilled the study restrictions. After passing the requirements, the participant read and signed the Informed Consent document, describing the purpose, tasks, risks, and benefits to participating in the experiment. Then, a paper-based demographics questionnaire was given, asking for the participant’s age, sex, highest level of education, military experience, current health state, proficiency using a computer, and video game proficiency. Next, the participant was randomly assigned to one of the three conditions: control, excessive positive feedback, and competition. The participant viewed a narrated slide presentation that familiarized him or her with the virtual environment interface and the experimental task (i.e., monitoring the virtual unmanned ground system (UGS), selecting the classification button, and clicking on the desired target). Following the interface training presentation, the participant completed a scenario, allowing him or her to practice the detection and classification experimental task. The stimuli of this scenario were colored barrels, unrelated to the additional tasks of the study, to avoid priming effects. Next, the participant viewed another set of presentation slides describing the pre-test scenario task. In the pre-test scenario, the participant was required to monitor the virtual UGS, as well as detect and classify virtual agents that exhibited aggressiveness or nervousness based on prior knowledge and experience. Following the pre-test scenario, a narrated presentation on behavior cue analysis training content was provided. The presentation covered the purpose of behavior cue analysis, the Kinesics domain, and described each target behavior cue with its associated classification. Example photographs demonstrating each target behavior were also included. Towards the end of the presentation, instructions on identifying and classifying the kinesic cues in the following scenario vignettes were provided. If the participant was assigned to the competition condition, then a three letter identifier (e.g., KAB) was requested. This was used as a way for the participant to identify his or her score at the end of the study. All initials and scores remained anonymous. A leaderboard listing past scores was shown. All participants completed four scenario vignettes. The scenarios were given in a randomized and counterbalanced order. At the end of the four scenarios, a second leaderboard with the participant’s three letter identifier and score was shown to each participant in the competition condition. A final narrated presentation introducing the post-test scenario was then provided. The participant completed the post-test scenario. Finally, the participant was debriefed and dismissed.

3 Results

The normality of the distribution of scores was assessed for detection and classification accuracy, using the Kolmogorov-Smirnov statistic. Results showed a violation of the assumption of normality for both detection and classification accuracy. Due to the absence of a normally distributed curve and to better meet the assumptions of normality, the data for this experiment was transformed. The data showed signs of a negative distribution, therefore, a reflect and logarithm transformation formula was used.

A one-way between-groups analysis of variance (ANOVA) was conducted to examine the impact of game-based strategies on detection and classification accuracy. Participants were divided into three groups according to the applied instructional strategy: Group 1: control, Group 2: excessive positive feedback, and Group 3: Leaderboard. There was no significant difference between detection and classification scores. While the results are not statistically significant, the detection and classification accuracy means can be examined for practical significance. Table 2 displays the means for each group.

Table 2. Transformed detection and classification accuracy means
Full size table

To interpret the transformed means, the lowest value becomes the highest reported value. On the other hand, the highest mean value is interpreted as the lowest score. As a result, the detection accuracy performance score for excessive positive feedback group was the highest (M = .67), followed by the control group (M = .68), then the leaderboard group (M = .70). The excessive positive feedback group also had the highest classification accuracy performance score (M = .85), however the leaderboard group (M = .91) performed better than the control group (M = 1.00).

Additionally, the influence of game-based strategies on the percent change in performance scores (i.e., detection accuracy and classification accuracy) from the pre-test scenario to the post-test scenario was assessed with a one-way between-groups ANOVAs. There was no significant difference in game-based strategies for detection and classification accuracy. However, the results support the previous findings for excessive positive feedback and its impact on performance. Overall, these results provide a foundation for investigating the role of excessive positive feedback.

4 Discussion

Although the results in this experiment are not significant, there are practical reasons to examine the means. As previously stated, the excessive positive feedback group had the best performance score. A possible explanation for this outcome could be the presentation of the feedback. Excessive positive feedback is descriptive, intended for the learner to know exactly how he or she is performing and what needs to be improved. The goal of this feedback is to improve a learner’s performance without obtruding or detracting from the training task. Learners received immediate feedback during the excessive positive feedback condition, providing the opportunity to improve instantly. In contrast, the competition condition did not provide immediate feedback. The goal of the leaderboard is to measure the learner’s performance with an overall score. It offers a summary of the learner’s performance, but it does not guide the learner towards improvement. The learner has to wait until the end of the scenarios to receive a score. Finally, investigation into the role of excessive positive feedback and competition should examine the impact of psychological constructs (e.g., engagement and motivation) on performance. Specifically, how these measures would contribute to the previously stated research findings.

5 Limitations

An inconsistency between the training slides and the experimenter script was identified that may have contributed to the participants’ performance scores. In the training slides, the instructions indicated that the leaderboard would be shown after each vignette, but it was only shown before the first vignette and after the last vignette. The participant received no feedback in between vignettes, therefore there was a limited capacity for improvement. Future research should incorporate the leaderboard after each vignette to examine whether there is an improvement in performance.

6 Conclusion

This paper incorporated instructional gaming strategies in a GBT environment to assess performance of kinesic behavior cues. Overall, this research effort provides insight and offers recommendations to the training and education community. One future recommendation involves the investigation into the rate of feedback (i.e., number of times feedback is presented) and its impact on performance. Another recommendation would be to explore areas of practical application (e.g., local and state law enforcement agencies, training and educational domains, etc.) to determine the effectiveness of the instructional strategies. Finally, despite the limitations confronted with the leaderboard, future recommendation into the full examination of the leaderboard as an instructional strategy is warranted to understand its application on Behavior Cue Analysis research.