1 Introduction

When a change is introduced into a system, especially a complex system, it can trigger ripple effects and interact with existing system dynamics in ways that produce emergent behaviors and capabilities [1, 2]. This holds true for the introduction of new technology into a training program, as a training program can be considered a system. In this paper, we discuss the introduction of the Live, Virtual, Constructive (LVC) training paradigm and associated technology into live air combat training. The LVC training paradigm involves augmenting a live training environment using virtual and constructive (VC) entities. Virtual entities are controlled from simulators operated by trained domain professionals—i.e., an aircraft simulator operated by a trained pilot or an intelligence platform simulator operated by intelligence professionals. Constructive entities are computer-generated, controlled by a software program and often with oversight from a person who adjusts their behavior as a scenario unfolds.

The primary appeal of including VC entities in live air combat training events is that they will allow aircrew to interact with greater numbers of adversaries and within an artificially enlarged training space. LVC technology also may offer additional benefits, many of which may emerge only as the training technology begins to be used. Likewise, the technology could have negative training impacts that only emerge over time and use. This paper describes the partial results of research conducted to predict potential interactions of LVC technology with existing Navy air combat training practices, capabilities, and culture. Specifically, this paper focuses on a portion of the research that addresses ways LVC technology might be used to benefit aircrew proficiency beyond the basic goals of virtually expanding the training range and increasing the numbers and capabilities of adversaries in training exercises. Research results are expected to guide the design and implementation of an LVC training system for air combat.

2 Method

2.1 Participants

Thirty-six participants were interviewed over the course of the study. Twenty-two were active-duty F/A-18 pilots reporting an average of 1310.5 flight hrs (SD = 460.9). All but one was qualified to instruct. Five participants were active-duty Naval Flight Officers who either worked as Weapons Systems Officers (WSOs; n = 4) or performed command-and-control on the E-2C Hawkeye (n = 1), with an average of 1074 h (SD = 270.1) in their position. Six participants were former military aviators; five of these were fighter pilots, one was a former WSO, and one had been both a fighter pilot and a WSO. These interviewees had an average of 1938.0 flight hrs (SD = 1417.0). Other interviewees were an air intercept controller with 9 years of experience and two military modeling and simulation experts.

2.2 Data Collection

Informed consents were obtained from all participants except the two M&S experts, who considered the interview to be a discussion among colleagues. Permission to audio record was obtained for 19 individual participants and one group interview of five participants. (Nine individuals and the group were interviewed in facilities where recording devices were not allowed). When audio recording was not permitted, researchers took detailed notes. Audio recordings were transcribed and interviewees were given the opportunity to review the notes and transcripts and offer corrections.

Semi-structured Interviews.

One-hour semi-structured interviews based on the Critical Decision Method (CDM) [3] were conducted with two groups of five active duty instructors (one instructor participated in both groups) and 10 individual participants. Participants were briefed on the LVC training concept and the Navy’s goal of using it to improve training efficiency and virtually extend training ranges. They were then told that the purpose of the interview was to identify potential training concerns (ways LVC technology might interact negatively with current training practices), training benefits (ways LVC could enhance or supplement current training), and training hazards (ways that LVC might negatively impact training safety if the LVC system was to be implemented into naval air combat training without mitigation).

Some interviews followed the CDM protocol closely. In these, researchers asked the participant or one participant of a group to recall a memorable and challenging event that they experienced during a past air-to-air training exercise. The participant was then asked to give a verbal “walkthrough” of the event and, once he or she finished, researchers posed (to the individual or group if a group interview) “what-if” questions about the potential impacts of LVC technology on the training event that had just been described (e.g., “what if the adversary pilot(s) had been constructive?”).

Other interviews were only loosely based on the CDM. In these, interviewees volunteered their concerns about training impacts and hazards related to flying with VC entities in the live environment. The researchers prompted them to describe how each concern could manifest within the context of a training exercise and, if possible, a past event in which a similar problem had occurred.

Discussion-Based Data Collection.

Following assessment of the interview data, additional data were collected using a discussion-based approach. Discussion sessions of approximately one hour (30 to 90 min) were held with one group of five participants (one WSO and four pilots) and nine individual participants. At the beginning of each session, a respected, high ranking Navy reservist and former expert F/A-18 pilot gave an approximately 20-min, in-depth description of the LVC paradigm and ways it could be used to support Navy air combat training. He concluded the description by identifying a challenge identified in the semi-structured interview data assessment and asked the participant(s) what his/their thoughts on the challenge and resolving it were. The participant or group’s response was followed by a discussion of various solutions and their pros and cons for the initially posed challenge and then for a number of other challenges that were brought up by the expert pilot. During individual interviews, research team members also suggested challenges for discussion.

2.3 Data Analysis

Identification of Potential Hazards and Training Concerns.

Thematic analysis [4] of the interview data was performed to identify and categorize excerpts about potential LVC hazards, training concerns, strategies for avoiding hazards and training concerns, and training opportunities. Excerpts addressing these topics were assigned a code corresponding to the specific hazard or training issue being described. As excerpts were identified, the set of topics and corresponding codes expanded and previously coded data were reassessed in light of the new codes.

Coded data extracts, labeled by participant number, were reorganized by code so that all data related to a given hazard or training issue could be considered together and synthesized. Each set of grouped excerpts was organized into contrasting positions, as applicable. (Participants did not always agree on how features of LVC training might affect safety and training).

Two naval air combat experts separately reviewed the identified hazards and training issues, along with the associated interview data, during meetings with members of the research team. (The experts had 2,200 and 2,800 Navy fighter jet flight hours, including experience as adversary pilots; had been on three deployments; were qualified range training officers; and still flew F/A-18 s in the Navy Reserves). They elaborated on conditions that might cause each hazard and training concern to become a reality and described strategies that Navy air combat personnel use or could use to avoid them. Expert reviews were recorded in researchers’ notes and added to the categorized interview data.

For the present paper, the discussion-based transcripts were reviewed to identify potential design and implementation solutions. Because the solutions were provided by instructor-level personnel at the Naval Strike and Air Warfare Center (NSAWC) and were viewed as reasonable by the naval air combat expert who led the discussions, they are being treated as equally valid and worthy of further evaluation.

Potential hazards, training concerns, and recommendations for training opportunities were additionally proposed for evaluation by members of the research team.

3 Results

The current paper focuses on live air combat training opportunities made possible by the LVC paradigm. Four categories of potential LVC training opportunities were identified: increased scenario options, scaffolding, attentional support, and repetition. For all but one of the training opportunities in the latter three categories, potential negative impacts on realism were identified for some aspect of the live training experience. These emergent training opportunities and their potential effects on fidelity are presented below.

3.1 Emergent Training Opportunities

New scenario options that LVC technology makes possible include the two basic goals noted in the introduction:

  • Supporting the training of long-range combat operations and tactics by allowing VC aircraft to fly outside the physical boundaries of the training range (i.e., by virtually extending a given training range). (Source: U.S. Navy)

  • Presenting large numbers of adversaries. (Source: U.S. Navy)

In addition, LVC technology could potentially be used to:

  • Present advanced adversary aircraft profiles, including advanced aircraft electronic signatures, speeds, and aerodynamics. (Sources: Interviews, Discussions)

  • Expose aircrew to better perceptual cues for post-engagement and re-attack decision making by displaying realistic simulated radar data of an aircraft hit by a missile. (Currently, when aircraft are “killed”, they execute a series of exit maneuvers that initially can be mistaken for tactical maneuvering.) (Source: Discussion)

Potential scaffolding opportunities found in the LVC training paradigm involve using LVC technology to prepare aircrew to understand and learn more complex or difficult aspects of air combat. Three scaffolding strategies that air combat instructors might find useful are:

  • Exposing aircrew to scenarios of the next complexity level in the live environment and populating them with VC adversaries and some VC “friendlies” to reduce the risk of a mishap while aircrew acclimate to the new complexity level. (Source: Research Team)

  • Helping aircrew learn the perceptual patterns of different adversary set ups and configurations on their radar display by presenting them as if the radar system operated perfectly and then systematically degrading its detection range and track maintenance capabilities. (Source: Expert)

  • Giving instructors controls to vary the complexity of the training environment; for example, controls that allow them to vary the amount of noise and ambiguity in radar data, the sophistication of electronic attack effects, and the shape of the adversary’s weapons engagement zone. (Source: Discussions)

Four strategies were identified that involve using LVC technology to help aircrew focus on specific training objectives or on learning specific elements of the work and the environment. These attentional support opportunities consist of:

  • Teaching aircrew about weaknesses of different radar systems by laying the data from that system over simulated radar data of a perfectly performing advanced radar system. (Source: Research Team)

  • Allowing instructors to fly as a virtual (simulator-based) section (i.e., two-ship) leads with a less experienced pilot to draw pilot’s attention to aspects of the exercise that relate to the training objectives. The virtual presence of the instructor means the less experienced pilot would be able to focus on training objectives without worrying about running into the section lead. (Source: Expert)

  • Allowing instructors to upload performance-support graphics onto radar displays of less experienced pilots as sources of real time guidance and feedback about, e.g., optimal merge geometry, coverage by electronic protection sources, expected flow, the mission timeline, and adversary tactics and flow possibilities to anticipate. (Source: Research Team)

  • Practicing the air-to-air phase of an exercise (i.e., the merge) with VC adversaries to reduce the mishap risk while focusing on aspects of the merge that can be practiced in the absence of a live adversary. Flying with reduced consequences of error allows pilots to safely learn from mistakes (assuming mistakes are detected) and may reduce stress levels associated with the merge, which can also facilitate learning. (Source: Research Team)

Repetition opportunities are made possible by the LVC paradigm because VC adversaries can be recycled immediately and repeatedly to practice an engagement or mission phase and without drawing on costly fuel resources. Valuable uses of this repetition capability could be to support:

  • Refresher training while an air combat unit is between deployments. During that time, unit aircraft undergo maintenance and only a small number tend to be available to support training at any given time. Funding allocated for fuel during this time is another major limiting factor. (Sources: Expert, Discussion)

  • Practicing the merge. The low cost of using VC adversaries and the ability to quickly re-position them means VC adversaries can be used to support a quick succession of practice runs. The merge is complex, high stress, and difficult to master; thus, repetition in the live LVC training environment could contribute to the proceduralization and automatization of the basic skills involved. (Source: Research Team)

3.2 Ripple Effects: Fidelity Trade-Offs

The above scaffolding, attentional support, and repetition strategies represent a variety of ways instructors could leverage the LVC paradigm and technology to train more strategically and efficiently. However, many were associated with trade-offs in the realism of certain aspects of the live training environment (see Table 1). Of the realism tradeoffs, those in Rows 3–6 of Table 1 were explicitly identified as concerns during interviews, with multiple pilots raising each. Artificialities described in the other rows are consistent with and derived from the explicitly identified concerns, as well as with safety concerns raised about introducing other artificialities (e.g., an LVC on-off switch in the cockpit) into the live environment.

Table 1. Emergent training opportunities and associated impacts on training experience realism
Full size table

4 Discussion

The LVC training paradigm has the potential to increase the fidelity of certain aspects of training scenarios and even the training environment (e.g., by showing a realistic representation of a shot-down aircraft); however, a number of training opportunities made possible by the paradigm could negatively impact some aspect of the realism of operating in a live training environment (e.g., the ability to see adversary aircraft by looking out the cockpit window). Since live air combat training is considered the last stop before real combat, fidelity in that environment is viewed as highly desirable: most interviewed aircrew asserted that interacting with VC entities should involve the same flow and interplay of cognitive activities as interacting with real aircraft. This means, that, for example, there should be no LVC-off option to consider activating nor VC entities in the merge phase when pilots are supposed to be looking out their window and not at their displays. This is consistent with the importance researchers place on cognitive fidelity: To support the acquisition of expert, combat-ready levels of perception, decision making, and response fluency, changes to live air combat training should not change the cognitive work such that what is practiced is no longer the work to be performed in the operational setting [5].

Also critical to proficiency is the availability of corrective feedback [5]. Aircrew must be able to detect when their performance would be unacceptable or unsafe in the operational environment—they need to develop a sensitivity to the selective pressures of that environment. They need to be able to test the boundaries of effective and safe performance and recognize when they have been reached.

It can be difficult to know what aspects of the rapidly shifting, information intense air combat environment can be reduced in fidelity without changing cognitive work or critical feedback in some way. However, aircrew would not need to do all their live training as LVC training; nor would all LVC training have to be the same. This raises the question of whether cognitive-work-changing artificialities are important to avoid if they (a) are recognized by the aircrew as artificialities and (b) will not exist in all live training and practice. In other words, how much is proficiency affected by intermittent exposure to an acknowledged artifact in the midst of extended practice without the artifact?

Yet another research question is whether expertise contributes to a resistance to negative training that might be produced by artifacts. Research suggests that artificialities, (the use of simplifications and metaphors to explain complex system dynamics and relationships) can produce misconceptions that persist and may never be recognized as misconceptions and corrected [6]. Based on these findings, new learning might be more susceptible to artificialities in the training environment, compared with fluency development at the upper reaches of proficiency. Possibly even new learning can be rendered immune to artificialities, however, if the artificialities are varied across learning experiences. Thus, if a metaphor used to explain a system relationship is alternated with other metaphors or with a range of learning materials and experiences that address that relationship, misconceptions should be detectable and correctable.

In the absence of varied learning experiences and opportunities, fluency at the upper reaches of proficiency is likely to be just as prone to negative training effects as new learning. In other words, at all levels of proficiency, it is likely worthwhile to take advantage of training opportunities that benefit some aspect of expertise acquisition, even if their artificialities detract from another aspect, so long as they are used in combination with training opportunities not featuring the same artificialities.

Faced with practicalities such as resource limitations, instructors will not always be able to provide a variety of training opportunities. It would be useful in these instances to know which would be more valuable—practice and training in as high-fidelity an environment as possible or practice and training that includes learning aids such as scaffolding, attentional support, and other instructional interventions at the expense of certain aspects of realism and cognitive fidelity. The above arguments for the importance of cognitive fidelity and corrective feedback could suggest that realism is most valuable; however, it is nonetheless possible that certain artificialities could integrate into a high fidelity environment to support the cognitive work of learning.

In summary, the keys to augmenting learning in a live complex performance environment may be realism and variety. In addition, we hypothesize that instructional interventions that compromise realism can be more valuable than pure realism if there is variety in instructional methods, materials, and scenarios. To the extent that the realism of the cognitive work and feedback is preserved and a given training experience featuring a given artificiality is alternated with other training experiences featuring other artificialities, learning and proficiency development should benefit from training interventions made possible by the LVC paradigm and technology. Additional research is needed to assess this hypothesis and to better understand the ways realism distortions in the live training environment interact with variables such as level of expertise, variety of training experiences, and the intensity (time and effort) of the training experience.