1 Introduction

The complexity of today’s technical systems is constantly increasing. This results from the amount of functions and advanced capabilities of these systems. In general, enhanced machine capabilities—as automation—lead to an increased performance and/or relieve the user. But the technical system and its automation must be operated. That means, a user is involved in the supervisory-control process and has to deal with the system complexity. That is, as the capabilities of automation increase, the risk arises that the complexity due to the advanced automation is the weak point of the system.

Shifting tasks from the user to automation brings other problems. On the one hand, these are problems such as out-of-the-loop effects (Endsley & Kiris, 1995) or loss of situational awareness (Endsley, 1999) due to insufficient human involvement in the work process. On the other hand, these can be problems such as excessive workload conditions due to demanding cognitive tasks that remain with the human (Onken & Schulte, 2010), and workload peaks induced by the automation itself due to “clumsy automation” (Wiener, 1988).

Therefore, the question of the optimal “work share” between the user and automation becomes increasingly important. Only if the work share is well organized, the capabilities of automation are an advantage. The “classical” paradigm of sharing work between the user and automation includes a hierarchical relationship. The user delegates explicitly tasks or parameters, i.e., command, monitor, and, if necessary, correct the automation. If the automation fails, the user has the responsibility to correct it.

The “classical” paradigm makes the user the only instance of the work system that has the responsibility and the abilities to achieve the overall work objective. The focus of the hierarchically subordinate automation is limited to a sub-function. As a result, the hierarchically subordinate automation cannot prevent errors at the work objective level.

To compensate the disadvantages of this “classical”, hierarchical automation, we introduce a heterarchical, cooperative automation (Onken & Schulte, 2010; Schulte et al., 2016). Within this heterarchical relationship, the user and automation (as “cognitive agent”) cooperate as partners. Each partner pursues the work objective on his/her or its own initiative and the automation supports the user in his or her task. In particular, this compensates the disadvantage that only the user pursues the work objective.

This article describes this cognitive agent as associate system to support the user adapted to his or her mental state. The goal is to prevent workload peaks and human errors while keeping the crew “in the loop” to prevent out-of-the-loop problems (Endsley & Kiris, 1995), complacency, and automation bias (Parasuraman & Manzey, 2010). Therefore, the associate system takes the role of an artificial, restrained-behaving team member. For this role, Onken and Schulte describe a set of rules as basis for this restrained behavior (Onken & Schulte, 2010). The basic rule is to let the crew do their job as long as possible without the associate system intervening. However,

  1. 1.

    If the attention of the human is not on the objectively most important task, guide the human’s attention to that task.

  2. 2.

    If, nevertheless, the human is overtaxed, transform the task situation into one that the human can handle again.

  3. 3.

    If the human is in principle not able to perform the task and the cost of a failure would be too high, adopt the task as a last resort.

2 Task-based operationalization of the mental state

These behavior rules link the type and amount of support to the attention and mental state of the user. Therefore, the adaptive association system needs an operationalization of the human’s mental workload (Honecker et al., 2017). To be able to support task-specific according to these rules, it is not sufficient to know whether the workload is high, but it is also necessary to know why it is high, i.e., the causal task situation that leads to high workload. To pursue to work objective on own initiative, as described above, the agent needs knowledge about this common goal. Only this knowledge enables the agent to support the user in a task-specific manner, including also cognitive tasks. Therefore, as basis for the workload-adaptive associate system, we introduce a context-rich definition of mental workload, as described in (Schulte et al., 2015). It includes

  • The currently pursued work objective and the resulting tasks, which are necessary to achieve this objective, i.e., plan

  • The set of tasks, which the operator currently executes, i.e., activity

  • The demand on mental resources, which is necessary to execute the activity

  • Behavior patterns, which the operator typically shows during the task execution and variations from these patterns (Donath & Schulte, 2015)

2.1 Task model

We operationalize this definition of mental workload with tasks. Therefore, a hierarchical task model contains the necessary domain knowledge of the application, i.e., all tasks which the user can do (Honecker et al., 2016). This machine-readable task model enables the associate system to have sufficient knowledge of the domain, similar to the human user. This includes the demand on mental resources, which is necessary to execute a specific task, constraints, and relations between tasks. Since humans communicate very efficient by using tasks as expression for very complex situations, the associate system is able to communicate on the same efficient way with the user. It is straightforward, that the associate system uses the same efficient way also for the internal communication between different modules. In addition, the task model enables the system to know different variants of task sharing between the crew and the automation.

According to (Young et al., 2015), workload can be reduced by automating tasks. That is, increasing the level of automation means to decrease the involvement and therefore the task load of the human operator. The Levels of Automation (LoA) in our task model define how the associate system can reduce the crews’ task load by automating a specific abstract task higher (like the example of an aircraft system configuration task “SetSystemConfigurationTransit” in Fig. 15).

2.2 Plan

Basically, mental workload qualitatively follows the task load (O’Donnell & Eggemeier, 1986). As more tasks a user has to do, and as more difficult they are, the higher is the workload. Therefore, the imposed task load, i.e., the tasks, which the user has to do, is a part of the operationalization of mental workload (MWL). The task load includes the tasks which the user currently performs and additionally the tasks to be performed in the future, i.e., the plan.

Based on the tasks of the task model, the plan is a dynamically generated sequence of mission tasks ( , see example mission plan of a transport mission in Fig. 1), which fulfills the mission objective ( , see Fig. 1). Therefore, the plan depends on the objective and in addition on constraints of the domain.

Fig. 1
figure 1

Example plan (i.e., sequence of mission tasks) of a transport helicopter mission. From (Brand, 2020)

Full size image

2.3 Activity

The plan is the basis for the user’s purposeful activity. However, the actual activity of the user may intentionally or unintentionally deviate from the planned tasks. The activity is defined as those elementary tasks that the user performs in parallel at a certain point in time (Honecker et al., 2016). This set of tasks is an essential part of the context-rich definition of workload, since the workload and attention distribution are directly related to the activity.

For the associate system, the activity is a crucial information, since it represents the current working context of the user. The task context enables the associate system to support task-specific. In addition, the distribution of attention of the user can be derived from the activity. However, since it is not possible to directly observe mental processes, this activity can only be determined indirectly. During the execution of the activity, the user interacts with the system in different ways. These interactions include primarily manual interactions, such as button presses or control inputs; visual interactions, such as the visual acquisition of information; and auditory interactions, such as voice input or verbal communication. Based on these interactions and the knowledge on task of the task model, the system can derive the user’s activity.

2.4 Demand on mental resources

The execution of the plan requires mental and/or physical resources of the user. Some models describe the need for resources, such as visual, auditory, cognitive, or psychomotor resources, either as a single resource (Kahneman, 1973) or as multiple resources (McCracken & Aldrich, 1984; North & Riley, 1989). If the user executes several tasks in parallel, it can lead to resource conflicts, which also increase the MWL (Wickens, 2002). These models are suitable for a capacity-based representation of resources, but they do not link the resource demand to the causal task situation. But this is crucial when using the demand on mental resources as trigger for task-specific assistance. An essential aspect is therefore to describe the user’s activity with tasks and to link each task with its resource requirements. Therefore, the task model stores an eight-dimensional demand vector for each task, which represents the demand on mental resources for this single task. The associate system can derive a measure of the total resources demand for a task situation by combining all related demand vectors of the tasks with the conflict matrix of Wickens’ Multi-Resource Theory (Honecker et al., 2016; Wickens, 2002; Maiwald & Schulte, 2014). Due to the eight separately stored components of the demand vector for the different resources (visual-spatial, visual-verbal, auditory-spatial, auditory-verbal, cognitive-spatial, cognitive-verbal, response-manual, and response-vocal), our method provides information, which resource of the user leads to a workload peak. And, as mentioned before, from the activity determination, the associate system knows which task situation leads to that workload peak.

This context-rich representation of mental workload is the basis for adaptive interventions, which support the crew adapted to their mental state for specific task situations.

3 Concept and implementation of adaptive assistance

The following concept of adaptive assistance can be applied to a variety of domains where the users interact with automated, technical systems. The application “Military Manned-Unmanned Teaming (MUM-T) Missions” is well suited to evaluate the described concept, since pilots have to perform a wide range of highly time-critical tasks in dynamic missions. Therefore, the following section introduces the domain “Manned-Unmanned Teaming.”

3.1 Domain Manned-Unmanned Teaming

At the Institute of Flight Systems, the concept of Manned-Unmanned Teaming (MUM-T) is a well-established approach to improve future military aviation. Guiding multiple reconnaissance unmanned aerial vehicle (UAV) from the cockpit yields many advantages including information gain, flexibility, and safety. But the additional tasks of commanding the UAV and processing sensor information change the roles of the crew and influence the amount and organization of tasks in the cockpit.

In a conventional two-seated transport helicopter cockpit, the commander (pilot in command, PIC) is responsible for mission management (e.g., mission planning), communication, and system management. The Pilot Flying (PF) is mainly responsible for (manually) controlling the helicopter, navigation, communication, and system management. The additional UAV guidance tasks (see green arrow from PIC to UAVs in Fig. 2) change the commander’s role to the role of a mission manager, who has to mentally keep up with mission events at several, spatially distributed locations. The attention and effort required for this cannot be used for the guidance of the own manned aircraft. This changes the role of the Pilot Flying and increases his responsibility and scope of tasks for the manned aircraft (green arrow from pilot flying to “tool” helicopter in Fig. 2).

Fig. 2
figure 2

Work system of the Manned-Unmanned Teaming configuration in the human-agent teaming notation from (Schulte et al., 2016)

Full size image

Even though MUM-T brings operational advantages in complex and dynamic situations, especially these missions are challenging for the crew. The combination of such mission scenarios and the changed task structure in the cockpit can lead to high workload conditions and wrongly distributed attention. Situations in which the demand exceeds the available mental resources, for example, in time-critical multitasking situations, lead to reduced performance and human error.

As a solution to prevent high demands on the one hand and loss of situational awareness on the other, cognitive agents are introduced into the work system in two ways (see Fig. 2). Cognitive agents onboard the UAV enable task-based UAV guidance. The UAVs are in a delegation relationship to the operator and are able to receive and execute orders on an abstract level. This reduces the necessary mental resources for UAV guidance, so that it is basically possible to guide several UAVs from the cockpit (Rudnick & Schulte, 2017; Uhrmann & Schulte, 2011).

In addition, the following concept of task-based, workload-adaptive assistance is introduced into the work system as a cognitive associate system. As “third artificial crew member” on the worker’s side, it is responsible for preventing errors by avoiding workload peaks or incorrect attention distribution and correcting errors that have already occurred.

3.2 Deriving adaptive assistance

Figure 3 depicts the functional architecture of the associate system (“This is inside the little in Fig. 2 above.”). The process of inferring adaptive interventions consists of three phases. Before the associate system is able to identify the trigger for assistance—neglected task, (predicted) workload peaks, and critical events—it preprocesses the mental state (Phase ① in Fig. 3). Thereby, it uses the plan to project the mental state into the future, to

  • Know which elementary tasks are necessary to fulfill the plan

  • Identify situations of high workload in the future, which may occur during the plan execution

Fig. 3
figure 3

Functional architecture of the associate system: “This is inside the little in Fig. 2.” The process of inferring adaptive interventions has three phases: mental state preprocessing (1), intervention decision (2), and intervention execution (3). From (Brand, 2020)

Full size image

In the second step, the associate system identifies trigger for adaptive assistance (“critical states”) and plans the intervention (Phase ② in Fig. 3). To implement the adaptive assistance, the associate system uses the human-dialog interface, which is part of the human-machine interface. In addition, the associate system uses other (cognitive) automation (e.g., the mixed-initiative planner or the adaptive crew sensor interaction) to simplify task situations or adopt tasks via the automation dialog interface (Phase ③ in Fig. 3). For detailed information about the process of identifying trigger, the decision process, and stages of intervention, see (Brand & Schulte, 2017).

3.3 Phase 1: Identifying trigger for adaptive assistance

For predicting future mental states and identifying resource conflicting task situations as well as neglected tasks, the associate system refines the plan—which is part of the “input” mental workload (see gray box “Mental Workload” in Fig. 3)—to a detailed plan on elementary task level (which is the lowest level of the hierarchical task model). It uses the task relationships from the task model to identify all elementary tasks, which belong to a specific mission task. Figure 4 shows the mission task EnterHOA, which is the task of entering a helicopter mission area, and its subtasks after the refinement process.

Fig. 4
figure 4

Excerpt of the task model displaying the hierarchical sub-task relationships of the mission task EnterHOA. The green boxes indicate completed; the red not completed tasks. If the associate system adopts a task, the box becomes magenta

Full size image

Because the activity determination expresses the activity on the same level of elementary tasks, the associate system is able to match the activity with the detailed plan and check the completed tasks (green task boxes in Fig. 4). All tasks which are planned, but not executed timely, are neglected tasks and trigger for adaptive assistance (red task boxes in Fig. 4).

In a second step, the associate system schedules all elementary tasks. Therefore, it uses the execution time and task constraints from the task model. This results in a task timeline, as Fig. 5 shows. The associate system simulates the execution of all future task situations and estimates the needed demand on mental resources (orange graph curve in Fig. 5). A situation in which the crew has to perform many tasks in parallel and therefore their demand on mental resources exceeds a threshold, is a workload peak (red marked period in Fig. 5) and trigger for adaptive assistance.

Fig. 5
figure 5

Timeline of the mission task EnterHOA and its subtasks (the upper eleven blue task boxes) and the mission task PlanMission and its subtasks. Due to many parallel tasks, especially induced by the PlanMission task, the associate system predicts a workload peak (red marked time period). From (Brand, 2020)

Full size image

The third trigger for adaptive assistance is critical events, like changes of the tactical situation, failures of the helicopter systems, or threatened UAV. The associate system cannot predict such events. However, if they occur, it supports the crew by guiding the attention, offering a possible solution, or directly implementing a solution. The following chapter explains which stage of intervention (which is the “amount of help”) is appropriate.

3.4 Phase 2: Deriving adaptive interventions

The process of deriving adaptive interventions aims to find the appropriate level of assistance for a given problem by taking the workload, attention, and criticality of the problem into account. The basis for this is the set of rules, mentioned in the section 1. To implement the restrained behavior, the associate system follows the decision process in Fig. 6: first, it traces back each trigger (gray boxes on the left side) to its causal task(s). After comparing this task with the current activity, the associate system infers if the crew works on the solution (Question A in Fig. 6). If the crew is currently not solving the problem, the associate system simulates, whether the crew can handle the current task situation including all other tasks, which are necessary to solve the problem (Question B in Fig. 6). Therefore, it combines the demand vectors of the hypothetical activity using the same method as for the resource assessment, described in Section 2.4. If the crew can handle the entire task situation, the associate system guides the attention to the problem (Stage 1 on the right side in Fig. 6), but if not, more assistance is necessary. If the crew is already overtaxed, or any additional task would overtax them, the associate system simplifies the task situation by adapting the human-machine interface (Stage 2a in Fig. 6) or changing the level of automation (Stage 2b in Fig. 6). Only if the problem poses an extreme or high risk (Question C in Fig. 6) and the crew cannot handle it, the associate system is allowed to adopt the problem-solving task(s) (Stage 3 in Fig. 6). We implemented this decision process with the cognitive framework Soar (Laird, 2012). For detailed information about the decision process and its implementation, see (Brand & Schulte, 2017).

Fig. 6
figure 6

Decision process of determining the appropriate stage of intervention (bold boxes on the right) based on the related trigger (gray boxes on the left) and the mental state of the crew. From (Brand, 2020)

Full size image

3.5 Phase 3: Stages of adaptive interventions

After identifying the appropriate stage of intervention, the associate system implements the different stages as follows. For guiding the attention to a problem, it uses the human-dialog interface. Therefore, it overlays a dialog box on the multifunction displays (MFD) of the cockpit (see left image in Fig. 7) and highlights all related objects on the MFD. In addition, it can simplify the task situation by adding a “shortcut button” to the dialog box, which implements the proposed solution with one button click (second image from left in Fig. 7). This “shortcut button” is also available via the helicopter control stick. Another possibility to simplify the task situation is to change the level of automation, e.g., to increase the level of automation of the mixed-initiative planner to simplify a planning-related problem. For critical situations, the associate system can adopt task(s) like a forgotten landing check (right lower image in Fig. 7), if the human is not capable of handling it. The MFD displays the information on a “history list of adopted tasks” (right upper image in Fig. 7). In addition, a speech synthesizer announces the adopted task (e.g., Set System Configuration Landing) as a human teammate would do in this situation.

Fig. 7
figure 7

Stages of intervention, escalating from left to right: Attention Guiding (1), Task Simplification (2a and 2b), and Task Adoption (3)

Full size image

4 Evaluation

To evaluate the associate system, we conducted a human-in-the-loop experimental campaign with trained military helicopter pilots in our helicopter mission simulator at the Institute of Flight Systems (see Fig. 8). The simulator has an extensive mission scenario simulation for manned and unmanned ground and air assets based on VBS3Footnote 1 and a realistic helicopter flight model based on X-Plane.Footnote 2 The purpose of the experiment was to investigate the interventions of the adaptive associate system during realistic and very complex MUM-T mission scenarios. Therefore, in Section 4.2, we analyze two (of many occurred) situations of “near misses,” where the associate system corrected human erroneous behavior during the missions. In addition, in Section 4.3, we present and analyze the overall system rating based on 45 interventions examined in detail.

Fig. 8
figure 8

Helicopter mission simulator at the Institute of Flight Systems, which we use to implement and evaluate the associate system for MUM-T missions

Full size image

4.1 Experimental design

The described adaptive associate system is highly integrated with the crew, the system, and the environment. The goal of the experimental evaluation is therefore to assess the presented concept by testing it as a highly integrated system in realistic and immersive scenarios. Accordingly, testing the associate system in a real system or a realistic simulation would be most expressive. The integration of the described system into a real aircraft requires a great time and financial effort or is not possible at all for certification reasons. Therefore, a realistic mission simulation offers a good compromise between unrestricted function integration and a realistic experimental environment.

Due to the realistic mission and flight simulation and the cockpit environment, the transferability of the results to the real world remains high (high external validity). However, the high complexity of such systems requires domain experts as test persons (i.e., experienced military helicopter pilots) and, in addition, extensive system-specific training. Due to the availability of the experts and the time-consuming experiments, the sample size of such experiments is typically small. In addition, the complex and dynamic missions lead to a second downside: a good internal validity is hardly achievable, since the large number of control variables makes it difficult to reproduce the results exactly.

For the reasons mentioned above (small sample size, high external but low internal validity), we chose an explorative approach. Therefore, we made no hypotheses or vary different configurations of the system. Instead, we use descriptive and sometimes explorative analyses to investigate when, why, and how the associate system intervenes to support the crew and how the crew rates these interventions. This approach provides some insights into when, why, and how the adaptive associate system supports—and how useful it was. Over a larger number of associate system interventions, also statistic evidence can be achieved. However, a representative sample is not possible due to the small number of subjects.

4.2 Specific adaptive interventions

One intervention affects the pilot in command (PIC), who is responsible for planning the mission and operating the UAV to ensure a reconnoitered flight route for the helicopter. In this situation, the PIC forgets to assign the route reconnaissance task for the next flight leg of the helicopter route to an UAV (highlighted in magenta in Fig. 9 and Fig. 10). In addition, he is involved in an ongoing route reconnaissance task of another UAV (see green frame in Fig. 9, which shows the tasks of the detailed plan for the next minutes). The two parallel tasks PlanMission and ReconRoute, starting at the red line in Fig. 9, lead to a predicted workload peak for the immediate future (red marked area, “now,” is at the left border of the red marked area). To relieve the task situation for the PIC, the associate system instructs the mixed-initiative mission planner to increase the level of automation (see also the planning automation levels in Fig. 15) and to propose the next relevant tasks on its own initiative. The planner proposes the forgotten route reconnaissance task (see dialog box in Fig. 10). Due to the urgency and criticality of this task, the associate system automatically accepts the planner proposal. The pilot rated this workload-adaptive intervention as very helpful and appropriate for this risky and time-critical situation (Brand, 2020; Brand & Schulte, 2018).

Fig. 9
figure 9

Prediction of the task situation in the near future (blue blocks are the tasks) and estimation of the future workload (orange solid line) with predicted workload peak (red marked area) and missing UAV task (PlanMission task in magenta frame). From (Brand & Schulte, 2018)

Full size image
Fig. 10
figure 10

Screenshot of the tactical map display with the planner proposal dialog box and the missing UAV reconnaissance tasks (highlighted in magenta) which are directly in front of the helicopter symbol and therefore very urgent. From (Brand & Schulte, 2018)

Full size image

Another intervention affects the pilot flying. In this situation, the helicopter takes off from a landing zone threatened by enemy units. From the south, enemy troops are approaching the landing zone (red symbols with circles in on the tactical map in Fig. 11). The commander has planned a helicopter route, whose first segment leads first to the southwest and then turns sharply to the north (white line with green border in Fig. 11). However, the takeoff procedure is difficult not only because of the threat in the south and the complex geometry of the route. In addition, the helicopter has been heavily loaded at the landing zone and the pilot must exceed the maximum continuous power limit to be able to take off.

Fig. 11
figure 11

Multifunction display of the right pilot with the tactical map on the left and the primary flight display on the right. The pilot’s green gaze point (including the local gaze measurement accuracy as a Gaussian distribution) is located at the helicopter on the tactical map.

Full size image

After takeoff, the pilot has to process the after-takeoff checklist immediately. This is important in the described threat situation, because completing the checklist also activates the electronic warfare system (EWS) of the helicopter. The associate system recognizes that the pilot has not yet processed the checklist, because two of three subtasks of SetSystemConfigurationTransit are still unprocessed in the detailed plan (see red subtasks of the complex task SetSystemConfigurationTransitManual in Fig. 12). In addition, the associate system uses the pilot observation to detect that the pilot is working on the two resource-intensive tasks FlyTakeoff and LocalizeHelicopter at the same time (see activity and total workload in Fig. 13).

Fig. 12
figure 12

Screenshot from the software module Plan Execution Monitoring, which shows a part of the detailed plan of the situation. The task SetSystemConfigurationTransitManual and the corresponding Check PerformAfterTakeoffCheckManual are both not yet fully processed. The landing lights are already off, because the pilot did not have to switch on the lights to land in the threatened area

Full size image
Fig. 13
figure 13

Screenshot of the program for workload estimation. Both, the takeoff procedure (FlyTakeoff) and the localization of the helicopter (LocalizeHelicopter) are very demanding for the visual-spatial channel of information acquisition and the spatial channel of information processing. How the adaptive associate system derives task situations and calculates the resulting workload measure is described in (Honecker & Schulte, 2017)

Full size image

This situation is classified as workload peak and thus triggers an associate system intervention. To find the appropriate intervention strategy for the intervention, the associate system searches in the detailed plan for tasks whose level of automation can be changed. As shown in Fig. 14 the abstract task SetSystemConfigurationTransit has three different levels of automation. In the detailed plan, the automation level SetSystemConfigurationTransitManual is currently selected. To simplify the task situation, the associate system selects a higher level of automation. The criticality of the situation allows the associate system to intervene even with the highest intervention strategy (take over the task). The associate system selects the highest degree of automation SetSystemConfigurationTransitAutomatic with the lowest human involvement.

Fig. 14
figure 14

Excerpt from the task model showing the abstract task SetSystemConfigurationTransit and its degree of automation

Full size image
Fig. 15
figure 15

Subjective rating of interventions determined from 45 interventions, which has been investigated together with the crews

Full size image

The pilot rated this adaptive intervention as supportive (“I felt supported by the associate system.” = 6 on a 7-point Likert scale, where 1 = not at all and 7 = absolutely); the intervention strategy was quite clear (“The intervention strategy was clear to me.”= 5) and adapted to the situation (“The intervention strategy was adapted to the situation.”= 6). The time of intervention was appropriate (= 6) and the intervention was reasonable (= 6).

These situations illustrate that the associate system adaptively supports the crew by taking the (projected) mental state and the criticality into account and that the pilots feel appropriately supported. Beside the investigation of situations of near misses, the pilots assess the overall performance and behavior of the associate system.

4.3 Results

The following results show the overall subjective system rating based on 45 associate system interventions. The subjects felt supported by the associate system in 84% of the cases and found 87% of the interventions useful. In 93% of the interventions, the type of intervention (intervention strategy) was assessed as appropriate for the situation, but in only 78% of the interventions the type (i.e., one of the three intervention strategies) was clearly understood by the subjects. In 78% of the cases, the time of intervention was appropriate, since in 67% of the cases, the subjects had sufficient time to solve the problem and 87% did not feel under any time pressure due to the intervention. The extent of the interventions was mostly appropriate, since only in 11% of the cases the intervention reduced too much work and in 14% of the cases the associate system did not support enough.

Overall, we conclude that the associate system is useful, and the adaptive interventions are beneficial. The system has been able to correct crew errors in many cases. For example, the associate system prevented the total loss of the helicopter several times (e.g., by extending the landing gear at the last moment before landing, or by automatically re-planning the flight path shortly before a threat was detected). Thus, task-specific support, in which the associate system cooperatively works on tasks, is helpful. Assistance that provides general support without knowing the task context is not able to correct such situations.

5 Conclusion

In this article, we presented the concept of a workload-adaptive and task-specific associate system. The key concept of the associate system is the cooperative relationship to the user. It behaves restrained and helps only if necessary and as less as possible to keep the human in the loop and to prevent typical pitfalls of highly automated systems, like out-of-the-loop problems, complacency, and automation bias.

The basis is a context-rich definition of mental workload. This complex measure describes the mental state of the user. For the operationalization of this complex measure, we use tasks of a machine-readable task model. With this definition of mental workload, the associate system is able to project future task situations to identify all pilot tasks which are necessary to reach the mission goal. In addition, it predicts situations of high workload in the future and eases them proactively by using higher levels of automation. Beside neglected tasks and workload peaks, the associate system uses situation changes as trigger for assistance. Based on these triggers, the associate system plans which intervention strategy is suitable to support the user. The concept was realized as a closed-loop associate system and integrated into the helicopter research simulator of the Institute of Flight Systems.

The overall system was evaluated in a pilot-in-the-loop experimental campaign with military helicopter pilots in Manned-Unmanned Teaming missions. The results show that the task-specific support in complex mission scenarios is beneficial and useful. The associate system intervened specifically for 37 different task types distributed over the entire task spectrum of a military helicopter crew.

The concept of supporting cognitive tasks workload adaptively recovers human errors like neglected tasks and relieves time-critical task situations before they occur. The pilots rate the interventions of the associate system as helpful and expedient. However, future improvements of the system should address the transparency, e.g., by pre-announcing the human-agent task sharing.

We implemented this concept in the domain of military aviation. But the concept is not limited to this domain. It is transferable to other domains, where a human operator collaborates with highly automated systems, e.g., civil aviation, highly automated driving, or power plant management.