1 Introduction

Traffic management is a central task to ensure safe and efficient services to passengers and transport of goods. In domains such as air traffic management (ATM), vessel traffic service (VTS), and train traffic management (TTM), operators oversee complex time-dependent traffic developments, involving individual and multiple moving vehicles, where the ultimate and high-level goal is to avoid conflicts and accidents. From a traffic control operator standpoint, the individual vehicle process unfolds with a certain degree of autonomy since the driver is in direct control and actuates the motors and steering of their vehicle. Additionally, related processes may be running at the same time, with various degrees of interference, overlap, and need for guidance from the control room. Hence, issues can emerge that, from a traffic management perspective, deviate from what is a desirable traffic flow. Consequently, these deviations—whoever monitors the process (humans or automation)—need to be identified and solved.

To improve efficiency and safety, automation is used increasingly in traffic control—but perhaps not to the extent to which one might think. Automation is introduced slowly, since it is challenging to build automation that can work with an operator and at the same time pass the strict safety requirements of traffic control domains. It is a well-known problem that “automation surprises” emerge when adding automation. For instance, “the designer who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate” (Bainbridge 1983, p 775). The answer to automation surprises, however, is not simply to add more automation or to avoid automation altogether. For example, it has been shown that focused attention on one process can distract an air traffic controller (ATCO) in a training scenario, to miss an unwanted development on the runway (Lundberg et al. 2016). In a related study, it was exemplified that just adding traffic conflict identification tools does not guarantee that the conflicts are attended to and found, since the traffic picture constantly shifts and since the operator also needs to maintain their independent traffic scan patterns. This introduces an issue for operators in their strategic use of automation—it is not always as simple as automating a task by shifting it over to the computer (Lundberg et al. 2015). Furthermore, research in the VTS domain has indicated that although low-level automation makes work more pro-active, it can be beneficial at lower traffic densities, but provides less support in time-critical situations (Aylward et al. 2020). With higher levels of automation, such as autonomous ships that are remotely controlled, the whole system might have to work differently, e.g. with pre-defined routes and increased tactical VTS work (Relling et al. 2022). Finally, it should be noted that even different operations that go under the same label, for example, VTS, may in fact differ markedly when looking at the work as done (Praetorius et al. 2015).

When higher levels of automation for traffic monitoring and process control are introduced, having a deep understanding of how operators control processes—from a cognitive standpoint—is important to be able to appropriately design decision support. Particularly, understanding time and operator time management and the relationship to cognitive demands and control strategies is a challenge not fully solved.

Since designing automation support is problematic, it would be advantageous if solutions could be re-used across control domains. This challenge is what motivates the current study and forms a key part of our discussion. However, the study itself poses a more basic question—how similar are these domains really in terms of the control activity? As a starting point for comparison, we first identified a “general” control challenge that on the surface seems to recur in all three domains. We then modelled this challenge from the perspective of operators, using a baseline scenario. Taking the perspective of the operator, in current ATM, VTS, and TTM systems—at times, there are situations that may become problematic, but that are yet not out of the ordinary. Not all these situations develop into adverse events but can still have negative consequences for the traffic system. In these situations, communication of information about the system state, the flow of decisions, and actions between the control system and the operator becomes crucial. For instance, at times, information indicating a deviation is visible on the displays, but there is an information flow problem. The operator may not notice the information, and the automation does not have the sensitivity to identify this issue, which can also be described as degradation of joint control. Although, in principle, this kind of event may occur in many domains, a key question is whether there are domain-specific developments that are sufficiently similar for a cross-domain approach to be viable. The similarities or differences may regard the operator’s physical and cognitive work environments, authority, or speed of development. Actual occurrences are rare, which is ruled out using observational studies. To instead devise a simulation to find out would require exactly the basic knowledge that we seek to find in this study—how can challenging control situations play out in different domains and how do they compare?

There have been quite a few studies on control processes, situation awareness (SA), cognitive work processes, etc. (see Sect. 2.2). However, only a little research has been devoted to compare domains and model temporal control processes. In this paper, we present a temporal analysis of work episodes in three traffic domains using the Joint Control Framework (JCF) (Lundberg and Johansson 2021). Based on the results of our analysis, we discuss possible ways forward to improve traffic management and control through automation support: considering one work episode per domain, selected to be challenging, yet not out of the ordinary, sharing some surface/assumed similarities (known/assumed before detailed analysis) between domains:

  1. (a)

    What aspects of the work processes are similar and what aspects are dissimilar?

  2. (b)

    What analytical notions were central in the comparison?

  3. (c)

    What are the implications for the interaction design of automation support solutions in each domain?

  4. (d)

    What is the potential for sharing (some) solutions across domains?

The article is arranged as follows: First, we describe the traffic domains and approaches to modelling work processes. The following part is a description of the data collection and the analysis procedures. Third, we present the JCF analysis for the three domains followed by a discussion on the comparison between the domains and the implications for the design of joint human–AI/automation work. Lastly, we conclude the article with some final remarks on the results and give some directions for future studies.

2 Background

The three traffic management services within this study all share the common goal of providing safe, efficient, and expedient traffic movements/patterns within a defined sector or area. The work to achieve this is performed by operators working at control centres. The air traffic control (ATC) centres, VTS control facilities, and TTM premises look—from the outset—quite similar. Typically, they have video walls and monitors depicting moving vehicles and their state on maps depicting airspace, fairways, or railway systems (Fig. 1). The situation displays show the location of vehicles, with status indicators in boxes or labels, and other important aspects of the environments that relate to traffic management. However, there are also differences among the domains such as the 3D freedom of operation of flight versus the 2D nature of vessel traffic and the 1D nature of trains, as well as the differences in authority (control versus information service) and differences in time requirements.

Fig. 1
figure 1

a En route air traffic control with the main air situation display (radar display). ©LFV press images, photograph: Peter Karlsson (top left); b VTS operator monitoring the traffic at VTS West Coast (top right); c TTCO monitoring traffic in a control centre (below)

Full size image

2.1 The ATM, VTS, and TTM domains

Air traffic control (ATC) is a service within the air traffic management (ATM) domain provided by air traffic controllers (ATCOs) in controlled airspace. The overarching goal is to maintain safe and orderly air traffic. The basic separation rules stipulate that aircraft shall be separated with 1000 feet or 5 nautical miles (NM). Although the overarching goal for ATC is to provide safe and ordered air traffic, safety always has a priority. Hence, the general monitoring of the traffic is primarily aimed at detecting conflicts between aircraft. However, airlines plan their flights to be as efficient as possible to save fuel. Thus, all actions taken by the ATCO may reduce efficiency for the individual aircraft, for example, by longer flown distance or flying at non-optimal altitude. ATCOs control the traffic to reach those goals. The ATCOs give clearances to aircraft to follow their planned routes and control deviations to avoid conflicts. ATCOs can also control traffic to enhance the traffic flow, for instance, towards airports using speed control and radar vectoring.

In en route ATC (area control), which controls all airspace above 9500 feet (the altitude limit applies to Swedish airspace), the main source of information is the air situation display where a synthetic radar image and flight plan data are presented in Fig. 1a. There is also a set of support tools, such as the medium-term conflict detection (MTCD), and safety nets, such as the short-term conflict alert (STCA). Flight strips for visualizing flights are not used in the Swedish en route system, all information is presented on-screen in aircraft labels, lists, and tools. The episodes analysed in this paper take place in an en route environment. In area control ATC, most information is acquired by the ATCO at a low level, mainly from the radar display and its associated tools such as vector tools. Even though some information is presented at a slightly higher level, such as in the conflict and risk display (CARD), it is still up to the ATCO to combine it with the rest of the information to get the overall picture. Communication is mainly performed via VHF radio. Automation in ATC area control is mainly focused on monitoring tasks and safety nets, such as the MTCD and the STCA. The MTCD uses trajectory predictions to detect conflicts. Due to uncertainties in for example wind forecasts, the quality of trajectory predictions is better on shorter time horizons. This means that possible loss of separation detected early by the system may not actually occur. The ATCOs must use their experience to judge whether an action shall be taken and when.

Vessel traffic service (VTS) is a service provided by vessel traffic services officers (VTSOs) to merchant vessels in port entrances and other areas that pose navigational difficulties. VTS is purely an advisory service, which means that the responsibility for the safe conduct of a vessel remains with the Master onboard. VTS aims to provide information to the marine traffic that enables efficient and traffic movements in the area. Due to its nature as a distributed system, the VTSO can only provide information and advice, but not concrete directions and instructions. Conflicts cannot be resolved by the operator, but a resolution can be supported through the information that is provided by the service. To ensure safety, the VTS pro-actively contacts ships that behave unexpectedly and potentially risky to check their intent versus the operator’s expectations.

VTS is supported by large-screen overviews of the traffic area, in which all traffic is monitored in real time (Fig. 1b). The VTS operator monitors the traffic in a pre-defined area in the VTS system, i.e., an electronic chart display integrated several real-time data streams, such as RADAR and AIS. A VTS system may also include CCTV cameras. The main tool for the provision of VTS is the VHF radio, which is used to receive reports from the vessels and provide all information necessary for the safe conduct of the traffic. This information may include but is not limited to the current state of the traffic in the immediate surroundings, future traffic meetings, or hydrometeorological conditions. In VTS, information is acquired from the radar and electronic chart display, as well as from the AIS transponder data. The VTSO can utilize tools, such as vectors, to predict single vessel movements, but generally the operators rely more on experience and expertise as active seafarers to assess the current traffic situation and provide information on the VHF (Praetorius et al. 2015). Low-level automation mainly provides alarms, such as alarms for shallow waters or pre-defined no-go areas. There are optional pre-defined alarms which can be added, such as alarms for dragging anchor. However, the current decision support does not provide predictive support for the traffic, more than vectors (based on course and/or speed over ground COG/SOG).

Train traffic management (TTM) in Sweden is a centralized remote-control process provided by train traffic controllers (TTCOs). The country is divided into eight regions, each one of them having its own train traffic control centre. In contrast to ATC and VTS, safety is not the primary concern in TTM in Sweden, since there are safety systems on low levels taking care of hazardous situations. The TTCO instead focuses on balancing efficiency for each train and fluency and capacity utilization from an overall traffic situation perspective. Train traffic management is carried out so that a TTCO manually ensures that switches and signals are set according to the planned route for each train. Small deviations and large disturbances make it necessary for TTCOs to reschedule one or several trains on a regular basis. It is in this real-time traffic management process that the TTCOs need to actively change the train traffic plan for one or several trains. They carry out these rescheduling tasks by continuously monitoring all trains in their allocated area. When there is a potential conflict, they strive to solve the conflict by replanning with such a good margin that the train drivers get the new information well in advance, which is before they reach that part of the track. Communication with train drivers is performed via telephone, but TTCOs most commonly are ahead of the trains so that the communication instead is mediated through clearance in signals and switchgears. In that way, the train drivers do not need to call up the TTCO over the telephone to ask for clearance in the signals.

In TTM, the TTCOs can see the incoming trains on large displays on the wall in the centre and on their individual workstation screens (Fig. 1c). Each region is then further divided into areas, and for each such area, one TTCO has the overall responsibility for all traffic in that area. In TTM, most information is acquired by the TTCO at a low level. A main source of information is the display showing the status of all the trains overseen by the TTCO. In addition, the TTCO uses a big paper sheet where each train’s planned route is graphically represented in a time–distance graph so that it is easy to see where different trains can meet (when there is a single-track area). Information from the signals along the track is shown as symbols on the screens in the train traffic control centre and on the workstation screens. The TTCO cannot see the exact position of the trains since there is no real-time update on the position. Instead, each track section (from some hundred metres to several kilometres) is marked as a blocked section for that train on the graphical displays on the screens in the control centre. TTM has a low level of automation based on visual signals in the interface on the screens. There are optional pre-defined automatic execution functions of several switches in a row that can be activated, but in most cases, these are inactivated since they interfere with the TTCOs mental time horizon or lead to actions for switching them off since these plans align with what the current situation demands.

2.2 Modelling and understanding control processes

To understand joint control in the three domains, a central issue is that they consist of a (set of) process(es) that the operator(s) need to keep up over time with and understand, to stay within safety and other desirable limits. To describe the processes that go on is an important step in understanding human interactions with automated systems (Lundberg and Johansson 2021).

In all three domains, there are vehicle processes that must be monitored to ensure that they work within limits both individually and together. This stands in sharp contrast to word processing, where the process is entirely generated through the typing actions by the operator, and it is entirely self-paced—if the typist stops, the words stop flowing. Furthermore, in the control rooms of all three domains, there is some degree of automation, at least at a low level, regarding information processing into integrated situation displays that are automatically updated. Automation can also exert control over the traffic processes, more or less independently of operator actions. This means that the work of automation becomes additional processes for the operator to monitor and control. Thus, these automated processes, their level of automation, and their interactions with human operators (Kaber and Endsley 2004; Nylin et al. 2022) must also be understood, as well as their degree of autonomy (Lundberg et al. 2021; Nylin et al. 2022). The division of work between humans and machines has a long history of being described on levels-of-automation schemes ranging from humans doing everything to machines operating without human involvement, with several levels of shared control in between (Sheridan and Verplank 1978; Vagia et al. 2016). These schemes are widely used in industry today, such as in maritime operations (Rødseth and Nordahl 2017). These schemes are, however, problematic, in that they de-emphasize how the interaction between humans and machines takes place on the in-between levels (Nylin et al. 2022).

Taking an operator’s perspective, keeping up with what goes on and with what is on the event horizon is central. This has been addressed using the notions of situation awareness, SA (Endsley 2015; Lundberg 2015; Stanton et al. 2017), or “construct” (Hollnagel and Woods 2005), or “edge awareness” (Stensson and Jansson 2014), regarding ongoing dynamic aspects of a situation to make decisions. Thus, the reason for studying SA in the first place is its centrality in human decision-making. This means that although SA can perhaps be studied in isolation, it can also be included as a context of decision-making (van Westrenen and Praetorius 2014). The central aspect of SA (or constructs) is the ability to process dynamic decision-relevant aspects of situations, so that they are considered in the moment of making decisions. This is what is meant by the notion of “awareness” in this context. This ability of maintaining the “construct” (i.e. SA) and constraints on that ability can be considered in the context of staying in control of oneself and of other agents, affecting and being affected by the quality of the control activity (Hollnagel and Woods 2005). Theoretically, the focus of SA research is mainly on the “known–unknown” (Denning 2006), to recognize situations, to gather information about them, and to look forward. Considering situation where people are figuring out, puzzling, and probing situations that have occurred to make sense of them, another strand of theory, sensemaking, is central. Broadly, according to Endsley (2015), sensemaking is more backward-looking and situation awareness is more forward-looking.

Both sensemaking and maintaining SA can itself be seen as a process where operators interact with processes, over time. Information pick-up + building an understanding is central to SA and sensemaking theory and can be represented as a loop (Endsley 2015; Lundberg 2015; Salmon et al. 2007; Smith and Hancock 1995). Considering control, information pick-up, and the generation of an understanding is important, but process-oriented decision-making and action are also important aspects. At the core, the basic control loop can be represented by the perceptual cycle (Neisser 1976; Salmon et al. 2007). The loop represents the cyclical and recurrent nature of control processes, even though actual events unfold on a timeline. Regarding time, it has for instance been shown in TTM that time frames are important to consider, since time pressure affects their strategies (Andreasson et al. 2019; Axelsson and Jansson 2018). Moreover, in TTM, with experience comes (perhaps unsurprisingly) the ability to visually focus more (fixate them longer, finding them faster) on task-relevant areas in the environment (Axelsson and Jansson 2022). To capture both time and the contents of control loops, it has been proposed, in the Joint Control Framework (JCF), to model, firstly, information pick-up in a process as perceptions directed towards a process and to model them on a human–process interaction timeline, a joint control score (Lundberg and Johansson 2021). In doing this, decision-making context is represented by modelling decisions and/or visible actions (to control or probe the process) on the score. The context of simultaneous processes that the operator works with can also be modelled as JCF scores (see Fig. 2). Thus, JCF combines a model of SA with a model of the situation, in terms of the temporal control processes that go on. Just as with SA, it is important to capture the view of the operator: what SA do they have and what processes do they see and work with?

Fig. 2
figure 2

A JCF score showing one control process with an abstraction hierarchy, starting with an observation on Level 3 (pattern recognition), then going up to framing (L6, what is going on), then forming goal (what goals to pursue, in this case both safety and efficiency, L5), considering these goals (L4), deciding on a plan (L3), and communicating with a specific vehicle to set its constraints (L1). If the capabilities of the vehicle, i.e. to carry out the plan, such as turn radius must be considered as well, then that would occur on Level 2

Full size image

When studying decision-making through a cognitive (systems) engineering approach, there is furthermore an emphasis on analysis through abstraction (Hollnagel and Woods 2005; Lundberg and Johansson 2021; Naikar 2017; Rasmussen 1986). Abstraction is achieved through a set of interacting layers of understanding (and of control) of processes. Abstraction goes from the concrete to more abstract notions, such as from the status of objects or actions to plans, to how the skills of operators or properties of objects interact with overarching plans, trade-offs, goals, and ideas of what goes on. Abstraction should not be seen as a linear process. As emphasized in cognitive work analysis (Naikar 2017), for instance, through decision ladder (Li and Burns 2017; Rasmussen 1986) and in the extended control model (Hollnagel and Woods 2005), abstraction is not necessarily a sequential bottom-up or top-down sequence of gaining understanding. Understanding and control may potentially jump between levels and start at any level (what occurs in various settings is an empirical question). Traditionally, in cognitive work analysis, the abstraction hierarchy starts with the notion of an effect goal, of a system, and then goes downwards. However, taking the centrality of the notion of framing into account, which is important in many control contexts, it has been proposed to add this as a sixth layer, framing, in the levels of autonomy in cognitive control (Lundberg and Johansson 2021); see Fig. 2. This sixth layer thus further emphasizes the subjectivity of operator views in control.

3 Method

We conducted a cross-domain qualitative case study in three traffic control domains: air traffic management, maritime vessel traffic service, and train traffic management (TTM). Data collection was done collaboratively with domain experts (i.e. the traffic process operators) at two separate full-day workshops. The goal was to address the generalizability both of control problems and tentative solutions across the domains. To ensure that similar control problems were addressed, a common scenario topic (a deviation in the process and process management where management could be improved) was identified jointly with operators in the first workshop. In the second workshop, the identified scenarios were collaboratively discussed and modelled in a simplified JCF score. The same workshop structure to explore it was used for all three domains, and the results were then refined and analysed by researchers using the full JCF approach.

3.1 Defining the scenarios

After the first workshop, identifying a cross-domain topic, a second workshop was used to discuss and collaboratively analyse operator work in more detail, from the operator’s perspective, through scenarios. The purpose was to generate rich, representative descriptions of work and process events leading up to a challenging control situation that is not out of the ordinary, and to also describe the resolution of such situations. As a starting point, basic traffic and work situation scenarios for each of the three domains were used. These were developed between the two workshops by the domain experts (see Fig. 3 for examples from the VTS and ATC domains). The traffic scenarios illustrated the traffic and work situations, as a sequence of images, in a form familiar to the operators. Even though a recording from an earlier eye-tracking study was used for the TTM domain, only screenshots were used as workshop material. This was done to create some distance between the particular events in the recording and the goal of the workshop, which was to discuss and describe representative situations, based on their own experiences. In this second workshop, three experts from each traffic domain were invited to participate in a focused discussion of the work episodes from their perspective (although for the ATC domain, one of the three experts could not attend). All the experts were experienced operators with more than 5 years of operational experience, and all but two (one from VTS and one from ATC) were actively working as operators.

Fig. 3
figure 3

Two of the screenshots used to support the VTS scenario discussion during workshop two. The vessel is approaching a critical turn in the fairway (a, top left), which she misses (b, top right) deviating from her course and heading towards shallow waters. The figure below presents corresponding examples from the ATC scenario. The first image shows the scenario when it starts (c, middle) and then when the critical situation between aircraft A and B is discovered by the ATCO, including the indication in the conflict and risk display (d, lower). The TTM scenario was displayed using a video sequence; however, due to sensitive content, it cannot be reproduced

Full size image

3.2 Collaborative JCF analysis to model control activities

The second workshop commenced with a presentation of the scenarios in separate sessions for each domain. The experts were asked to provide feedback on whether this situation could be considered as realistic and whether they had encountered it (or similar situations) in their work before to confirm that the description reflected challenging, yet not out of the ordinary, work. After this short introduction to the case(s), the experts were asked to use post-it notes to write down which actions and decisions they would take, if they were the operator in this scenario. They were also asked to explain what information an operator would need throughout the scenario and which piece of the current equipment they would use to obtain information.

After 10 min of individual work, the experts were asked to start a discussion and compare each other notes by creating a timeline with actions, decisions, information, and equipment on a whiteboard. The workshop used a simplified JCF (Lundberg and Johansson 2021) analysis approach to describe the scenarios, which we call the JCF outline form. It used only one timeline (score) to describe the whole scenario (see Fig. 4, from the TTM domain), with tentative placement on the timeline of the post-it notes (i.e. the joints: perceptions, decisions, actions), between the traffic operators and their traffic processes, at the LACC levels (see Table 1). The moderator in each of the domain-specific sessions helped the experts to create a joint picture of the scenario, including placement on the LACC levels. The discussions were recorded to support further analysis.

Fig. 4
figure 4

Example of the collaborative JCF analysis, from the TTM domain (text in Swedish). It uses the JCF outline form in which all joints are modelled in the same score. Post-it notes represent the joints: perceptions (yellow), decisions (orange), and actions (green), as well as the medium used (pink). Over the score, notes (pale yellow post-its) are placed to describe more overarchingly what occurs in the episode. The score was then analysed by method experts to identify different joint control processes. (Color figure online)

Full size image
Table 1 Levels of autonomy in cognitive control, outlining the contents of the levels in the three traffic domains
Full size table

The levels of autonomy in cognitive control (LACC) (Lundberg and Johansson 2021) is an abstraction hierarchy that contains the domain process (the object that needs to be controlled), the operator supervisory process (the subject that controls objects), and the automation/tools (the medium) with which interactions on that level could take place.

During the second part of the workshop, the domain experts were asked to present the scenario, including the timeline of actions, decisions, and information/equipment to each other. The aim of the discussion was twofold; on the one hand, it was used to identify further commonalities and differences across the domains, and on the other hand, it provided an additional source for the data collection for each scenario.

3.3 Refining the JCF analysis

The analysis was conducted iteratively. During the analysis, the positioning of joints on the score levels on the scenario timeline (placed tentatively during the workshop, Fig. 4) was refined, statements were clarified, and visualization of relations between scores was added. The first analytical step after the workshop was to take the notes, in the JCF outline from each workshop and refine them using the JCF Editor software. This was done separately by each domain expert author of the paper. This tool supports editing JCF scores, using the JCF notation (Lundberg and Johansson 2021; Nylin et al. 2022), including both joints and relations between joints on and between scores. Firstly, the subject–object relations in the scenarios were identified. This defines what scores can be described, e.g. ATCO–aircraft, VTSO–ship, and TTCO–train. Then, the analysis was refined by the distribution of the joints over the respective scores.

In the next step, any potential level misplacements were discussed and iterated by the JCF method expert author, together with the respective domain expert authors. Furthermore, in this step, five “convenience” joints were added (Table 2). Firstly, we found it useful to highlight the point where the deviation occurred (and first became visible), in its respective process (deviation, DE), and when the operator interprets and draws conclusions from the perceived information (interpret that, IT). Secondly, we added core values at the start of the scores, as a remainder during the analysis of the pre-existing overarching process goals. Third, we added two supervisory joints: “check joints”, to highlight actions that are communicative read-backs of information, and “supervisory procedures”, such as scanning of traffic, which involve many perception joints. This simplification was needed since we did not model visual traffic scan patterns in detail in this workshop. In this final step, central relations, such as transitions from monitoring to a process-in-focus, were also highlighted by drawing lines/arrows between joints in the scores.

Table 2 Core joints and notation convenience joints. (Color figure online)
Full size table

4 Results and discussion

The three scenarios describe for each domain, qualitatively, the most salient parts—from the point of view of the workshop participants—of what they saw as typical events. At the core of each scenario, the participants described a traffic deviation. In the ATC scenario, the issue was the late discovery of an aircraft heading towards a separation violation. In the VTS scenario, a late discovery of a ship deviating from the intended route was modelled. In the TTM scenario, the operator overlooked a closed stop signal (that should have been switched to “go”) that affected the traffic flow negatively (i.e., the train must stop and talk to TTM). Furthermore, the ATC and VTS scenarios included a distracting telephone call. Only the ATC scenario included a more active support system, the medium-term conflict detection (MTCD) tool.

4.1 A process taxonomy

During analysis, the work episodes were divided into processes, according to the taxonomy in Table 3. Each domain contains objects (ships, aircraft, and trains) and their respective processes, which are being monitored. However, they are monitored, from the operator’s point of view, as part of a main monitoring process, of traffic. In each domain, we focus on a single operator as the subject, in line with what the operators described. In the case of ATC, there is also an automation, the MTCD system, involved as a subject in one process.

Table 3 Brief description of the identified processes in each domain
Full size table

The problem with the deviations, in all domains, is that they are not discovered in a timely manner (in the view of the operators). The part in focus for our analysis is what happens from the discovery of the deviation until the implementation of a solution.

In the ATC case, the ATCO discovers the deviation during the general monitoring of the traffic and then confirms it by checking the MTCD information. A short sequence follows where a solution is implemented to avoid a separation violation. The solution then must be monitored to allow the aircraft to return to its original route.

In the VTS case, the detection during general monitoring is followed by several joints where the VTS operator builds his or her understanding of the situation by communicating with the vessels involved. After some time, the VTS operator has enough information to suggest a solution to the situation.

Finally, in the TTM scenario, the operator is made aware of the situation by an incoming telephone call from the train driver who has stopped for a red signal. The TTCO checks that it is safe to give green light and then does so.

An extract of scores from one scenario per domain is shown in Fig. 5. Each joint is represented as a small rectangle on a timeline. In all three scenarios, the information about the deviation has been available for some time without being noticed (Fig. 5, magenta points in magenta circles), and the information acquisition is made at a low LACC. Furthermore, the period from discovery by the operator of the deviation (brown points in yellow circles) and implementation of a resolution (green points in green circles) are shown. Note that the time scale is not similar between the scores. The complete scores, although in a slightly different visualization, are shown in “Appendix A”.

Fig. 5
figure 5

Comparison of the scores from detection of the missed situation (interpret that joints—orange circles), until a recovery action is implemented in terms of action joints (green circles). Magenta circles show deviation joints, i.e. when information on the deviation was available within the operators’ systems. Note the two detection points in the ATC score. At the first point, the deviation is detected by the MTCD tool, but not yet detected by the operator. The scores with the orange circles (interpret that joints) are the monitoring scores, while the scores with the magenta and green circles are the processes-in-focus, which is a result of the framing (in the monitoring processes). As can be seen, the patterns of joints in the processes-in-focus differ between the scores. (Color figure online)

Full size image

The possibility to visualize and analyse the temporal aspects of control shown in the results and complement other studies of control and control strategies within traffic control by contextualizing the situations. For instance, Kontogiannis and Malakis (2013) examined control strategies based on the extended control model (ECOM) (Hollnagel and Woods 2005) in case of control loss in ATC. Further, in a study on ecological interfaces (Borst et al. 2017), control strategies were illustrated as single processes with a starting point and feedback loops. However, like all circular or recursive models of control, they provide a description on a conceptual level, but do not show how it is implemented or the interplay between different control processes. Regarding performance, measurements such as response time have been correlated with task load (Friedrich et al. 2018) when studying the impact on SA by task load. This provides correlation, but not the control activity context. A timeline to illustrate ATC tasks in the control process has been used before (Karikawa et al. 2014). However, it was restricted to ATC instructions, which correspond to a sub-set of action control joints in JCF.

4.2 A comparison between domains

The results show that it is possible to compare joint control processes from different domains, and the comparison can reveal interesting phenomena, such as differences between processes-in-focus. In the three cases, the issues described as representative events by the domain experts emerge from the “natural” traffic processes, rather than being direct consequences of the control system and operators’ actions. The management of the issues is similar as the operators with their support systems are not attending to the issues immediately. Our analysis can show how this is the case. We shall now discuss this further.

A word of caution when considering the results is needed. Since the work here is based on operator workshops rather than on observations, the exact timing of events has not been in focus. However, the exact timing is not critical to this study. The main rationale for this choice of method was that even though the situations with process deviations are representative, they would be very difficult to achieve on demand in for example a real-time-simulation, especially to achieve comparable situations it in all three domains. Another aspect that needs a word of caution is that we have worked with one case per domain—considerable differences can still exist in other cases across the domains.

4.3 Similarities and process-in-focus framing

On an overarching level, the comparison revealed cross-domain similarities between the domains in the operator work process. These similarities regard three classes of cognitive work: traffic monitoring, resolution of the process-in-focus, and managing the whole. The latter includes managing the first two classes of work and any additional work such as system alerts/communication from others. The traffic monitoring aims at detecting/preventing constraint violations based on patterns in the traffic picture and on object process (vehicle) characteristics. Moreover, it includes the identification of processes-in-focus that the operators examine and then attend (see “Appendix A” for a complete JCF overview). A process-in-focus can contain one or several object processes, and the interactions between them can be central depending on the framing. Likewise, it can be entangled with other processes, even if that is not shown in our example. As shown in Fig. 5, the processes-in-focus comprise different cognitive joints in different patterns, which is further elaborated on in Sect. 4.5.

According to our analysis, critical identification is foremost done manually by operators, including framing, looking for object-related cues, and examining larger process patterns. Further, in addition to monitoring, the operators are alerted to issues by automation (e.g. the conflict monitoring tool in ATC) and by operators of vehicles (by the object processes, e.g. a call from a vessel reporting or requesting information). These alerts can also interrupt the overarching human traffic monitoring process, in all three domains. In less urgent situations, the operator can make trade-offs between different actions and goals, for instance, switching from one process-in-focus to another one. However, the late discovery in the analysed situations called for an immediate response, leaving little or no room for prioritizations.

The framing of processes-in-focus is subjective, which is a challenge from an interaction design perspective. The responsibility and ability to manage both classes of processes (monitoring and processes-in-focus) and to manage the whole work situation of monitoring and processes-in-focus requires split attention, for a single person. Furthermore, the systems have several monitoring services, ranging from active like MTCD that can detect conflicts in traffic to less active, for example, the situation displays of vehicle positions and relay statuses. If the (subjective) framing of a process-in-focus from the operator does not match how the monitoring system frames the situation, it can, in the worst case, lead to the systems and humans having different views of the situation and counteracting each other’s actions. Hence, both the humans and the (automated) systems supporting the human must possess a degree of flexibility and understanding of the traffic situation to create a common ground for collaboration.

In the studied situations, there is only one system (the human) working actively with the control processes (i.e. monitoring and controlling the vehicle processes). This results in the following overarching cross-domain issues for the operators:

  1. 1.

    Balancing monitoring and processes-in-focus, to be able to continue one of them while working on the other—this is challenging while having responsibility for doing both.

  2. 2.

    Managing interruptions, in the timing of communications between humans and other agents (automation/alarms).

  3. 3.

    Making information available between subjects and objects, which connects the perceptions, interpretations, decisions, and actions better, to work upwards in the LACC with control joints and automation capabilities that are better aligned.

4.4 Differences between the domains

Though similarities between the domains were identified, there were also differences. These regard risk escalation, information available to operators, how detection occurred, and in the control context.

  • Firstly, the delays in the detection of process deviations in the episodes seem to be similar between domains in that they do not immediately generate a high-risk situation. However, in ATC and VTS, this kind of process becomes riskier as time proceeds, with ongoing vehicle processes being off-track. In TTM, similarly, there is a disturbance to the train process, but there is no escalating risk since the process halts. However, the disturbance to traffic flow efficiency increases in TTM as the process is halted.

  • Secondly, the information present differed between the domains in that the ATC scenario had an active conflict detection system that presented information about the conflict. However, in all three domains, the situation displays contained information on the status of the objects involved.

  • Third, detection differed between domains. In this ATC scenario, the operator first checked the conflict during the resumed monitoring, noticed a possible worsening situation, and turned to the MTCD to confirm it. In VTS, it was identified during a resumed traffic scan, and in TTM, there was a call from the driver.

  • Fourth, the control context also differed between domains. In TTM, the operators shifted from pro-active work in setting up the system (e.g. traffic signals on the tracks) to communicate with drivers, which then changed to reactive voice-based communication. In ATC, the operators shifted from approving clearances in the premade plan to actively steering the traffic to solve the conflicts. In the VTS, the VTSO shifted from monitoring to closer examination of the situation of the vessel, helping them to recognize their deviation from the intended route. In VTS, the management of the focus–process episodes is much more complex than in ATC and TTM due to the polycentric control setting of the traffic system; that is, the operator does not exert control over the traffic as such, but each traffic participant controls its own actions. Thus, the VTSO needs to collaboratively interpret the situation, as well as to monitor its progress and anticipate potential escalations. In TTM, the action is quite quick, and the episode is short, but a follow-up episode to manage side effects follows. In ATC, the focus–process–episodes are short, but the operators must still monitor the upcoming conflicts until the aircraft has passed the critical positions. Further, the VTSO must actively interpret patterns and identify traffic participants (vessels) on the display, after a new call draws attention to a particular ship. This means the time away from the generic monitoring (or time sharing) is the longest in the VTS case.

4.5 Detailed comparison between the three domains

Examining the complete scores (“Appendix A”, Figs.6, 7, 8), we see a similar pattern; the perceptions (orange) are at lower levels, with implications (white) drawn by operators at higher levels. In all three domains, monitoring (top score, “Appendix A”, Figs.6, 7, 8) includes reading the status of physical objects (perception points at Level 1), interpreting this information at higher levels for unexpected/unplanned traffic developments (Level 2) and traffic pattern violations (Level 3). The exception is ATC where conflicts are also displayed at a slightly higher level (implementation, Level 2) by the MTCD. Decisions are made by the humans at higher levels. This discrepancy in levels means that the automation is not really doing much cognitive work and that humans do heavy cognitive work, discerning patterns, their meaning, and plans for action. Undesired effects are determined directly (decision points) in ATC and TTM. VTS has a vaguer frame, in that there are unexpected situations to investigate (VTS, decision point at Level 3).

Fig. 6
figure 6

ATC: ATCO monitoring traffic and solving conflicts. Firstly, solving conflict A + B and then B + C. Missing the development of A + B while working with a call from D regarding a direct route. (Pink joint at the start of the delay episode, third score from the top). Green (actions), white (supervisory), blue (decisions), orange (perceptions), and red (core values). (Color figure online)

Full size image
Fig. 7
figure 7

VTS: VTCO monitoring traffic, missing a deviation of vessel 2 while working with a call from vessel 1. (Pink joint at the start of the delay episode, bottom score). Green (actions), white (supervisory), blue (decisions), orange (perceptions), and red (core values). (Color figure online)

Full size image
Fig. 8
figure 8

TTM: TTCO monitoring traffic, missing a red light, for a train (pink joint at the start of the delay (D…) episode, bottom score). Green (actions), white (supervisory), blue (decisions), orange (perceptions), and red (core values). (Color figure online)

Full size image

In particular, the framing of processes-in-focus (top level, Level 6) and the criteria for doing so are all internalized by the operators. According to the LACC levels, this has consequences downwards for the relevant goals (Level 5), concerns (Level 4), patterns to look for (Level 3), the visibility of pattern instances (Level 2), and object characteristics that need to be seen (Level 1). Notably, concerns that pertain to (the amount of) risk or to trade-offs were not central in these episodes (Level 4, which can be seen by the lack of joints on these levels in the scores (“Appendix A”, Figs. 6, 7, 8). Such concerns could be present in other episodes.

4.6 Implications for the design of joint human–AI/Automation work

Based on our analysis and the identified issues, we have identified three avenues for future research in interaction design, which are discussed as follows:

Firstly, we find that a control paradigm shift would occur (perhaps it can only occur then) if the automation took over the responsibility for the main monitoring process from the operators. Then, the problem would shift from guiding attention in the case of a non-attended issue in a process to guide attention between processes-in-focus driven by the automated system. That would also bring issues such as (re)establishing situation awareness to each newly attended process-in-focus, to be able to exert control in it. Notably, it would require that the system was able to identify all the issues in the traffic (not just a sub-set), and the control scores in such a new paradigm would most likely differentiate substantially from what was found here, and this is therefore also a central issue for future research. Probably, it would also require new kinds of support for the operator. This broad capability is not how development has been historically done—instead, automation has been brought in with a focus on improving the management of specific issues, in specific kinds of focus–processes such as conflict detection and sequencing of traffic going towards/from squeeze points (airports).

Secondly, for systems that could only identify a sub-set of the issues, as the current situation is in ATC with its MTCD system, we instead see flexible automation (F-AUTO), i.e. automation that adapts its behaviour flexibly to the human’s situation, as a way forward. In the F-AUTO scenario, monitoring and/or addressing processes-in-focus would be shared by humans and automation. Therefore, interaction must be carefully designed, including consideration of the stiming of interactions, and an improved understanding by the automation of the operative priority of issues, as well as of what the operator is working on and how well that is going. A key issue, as mentioned above, that also holds for F-AUTO with humans as the main locus of control, is that all the processes-in-focus are identified and framed by the operators. When there is a frame mismatch between operator and system, humans and automation will no longer see the same processes-in-focus and consequently not see the same overarching characteristics of the traffic situation. Arguably, for the system to be able to prioritize how much attention it should call for when finding a lower-priority issue, it must also be able to identify a set of high-risk/high-priority situations and be able to determine whether the operator is busy working with these. The automated identification of domain-specific sets of lower-priority issues from traffic patterns, and processes-in-focus from operator work-patterns is a challenge for future research. From this study, we can see that the intervention/management parts for TTM and ATC in the missed interactions are short and clear, basic simple patterns for a computer to detect. The VTS case seems more complicated. Thus, for each domain, future work is needed to understand these interactions. Whether an F-AUTO solution should also be adaptable (Kidwell et al. 2012) by individual operators is a question for future research. This approach thus also includes substantial challenges. Much further research is therefore also needed to realize the F-AUTO scenario.

Thirdly, on its own, increasing the salience of specific items of information or adding alarms would be an option—but it does not seem promising from our study. This is because the MTCD presented the information but that was not seen as fully effective by the experts. However, in combination with F-AUTO, new notions such as soft visual cues (Nylin et al. 2020) could be used in display redesign for more effective communication. When working with attention guidance, however, we return to the issue of framing—if the automation does not cover all frames, caution must be taken in interfering with the operator’s visual traffic scan (where they use their own frames).

5 Conclusions

This paper analysed temporal work episodes for traffic control, from three domains, that were described by domain experts to be challenging but not out of the ordinary. The common joint control problem emphasized in all episodes was delayed control of a process deviation.

The analysis addressed four main questions on (a) similarities and dissimilarities between the domains, (b) central notions in the comparison between the domains, (c) implications for the interaction design of automation support solutions in each domain, and (d) the potential for sharing solutions across domains for automation support.

Cognitive control joints were identified and mapped into Joint Control Framework (JCF) scores. First, an overview JCF was collaboratively mapped by operators, and the JCFs were then iteratively refined and analysed by method experts. During the analysis, different control processes were identified, and patterns of control joints—from in the control processes and from the respective domains—were compared. Different control processes were identified, and the patterns of control joints in the processes-in-focus in the respective domains were compared to identify similarities and differences. The notions of control processes and cognitive joints were central to the comparisons. This approach of using the JCF framework to analyse the cognitive work in three different traffic domains and compare them proved to be successful and can be applied in the future similar studies.

The three domains had differences and similarities in how information is presented and in how monitoring and control are organized, which situates the episodes in different control contexts, summarized as follows:

  • The three domains contain individual vehicles (objects) with temporal processes that have their own drivers and constraints. They must be monitored individually but also require oversight and management as part of traffic as a whole.

  • All domains use displays that show a “now” picture where objects and their status are represented in an infrastructure (airspace, waterways, rail network). This object-centric view is at a low level of cognitive control. Objects are, however, not always in focus for the traffic operator.

  • In ATM and TTM, control is centralized. Operators in ATM in control of traffic give planned clearances in temporal sync with the process (soft constraints), whereas in TTM the rail network is configured ahead of the traffic (hard constraints). In VTS, monitoring is centralized. Information is given to traffic, often at specific time points. Control is however—in our Swedish case—decentralized, so that VTS can provide traffic information and ask questions but not give direct commands.

The analysis suggests that the core problem in all domains was the shifts between general monitoring and processes-in-focus. A complicating factor in this is that the (subjective) framing leads to different results in the different domains. To understand this problem and to suggest ways to improve work, we characterize cognitive work in the missed-control interaction episodes as follows:

  • Monitoring For oversight, to discover and address constraint violations, operators perform an overarching monitoring process to identify processes (i.e. conflicts) that need particular attention and interventions (processes-in-focus). Monitoring does involve examining object process status, but also examining traffic patterns and being contacted/alerted by object processes or by automation.

  • Processes-in-focus One or several object processes united by some common issue (e.g. a conflict), which will occur at some time point, perhaps ahead, become larger processes-in-focus. These can also include infrastructure (e.g. islands in VTS, rail infrastructure settings in TTM).

  • Framing The processes-in-focus are framed by the operators (at the highest LACC), and this affects how the situation is managed, illustrated by different control joint patterns. The systems differ in how framing is achieved and supported. In VTS, this is a shared activity among VTS and the involved ship(s) to agree on what goes on and how to address it. In ATM, the automated medium-term conflict monitoring system frames conflicts in terms of projections along flight plans. However, the ATCO cannot only rely on this frame, since the projected plan may change even if the aircraft are following given clearances, for instance, due to changing wind conditions. Thus, the ATCO must sync its view with the MTCD. Note, that although it did not play a part in this scenario, TTM has a conflict-centric view as well; however, it was still operated manually at the time of this study, on a sheet of paper.

  • Deviation A special temporal constraint related to process-in-focus is the time point when the identification of the situation is “late”, defined by a change in the nature of the process-in-focus (the deviation). This is perhaps most salient in the TTM domain, where the process changes from being fully mediated through the infrastructure (e.g. lights, setting of tracks) to being reactive and mediated through verbal communication with train drivers.

  • Monitoring vs processes-in-focus Finally, the operator must manage the whole work situation at the same time as the current processes-in-focus. This includes both regular work (clearances in ATM, information in VTS) and further monitoring to potentially identify new processes-in-focus.

We have discussed three ways forward to address the cross-domain issues found in this study, which could be addressed by future research. What solution to choose depends primarily—for these cases—on the ability of operators and automation to establish processes-in-focus. This in turn primarily depends on the ability of the system to establish and work within relevant frames:

  • A control paradigm shift would occur (perhaps it can only occur then) if the automation took over the responsibility for the main monitoring process from the operators.

  • Flexible automation (F-AUTO) could be a way forward for systems that could only identify a sub-set of issues in the overall traffic situation, as the current situation is in ATC with its MTCD system.

  • Increasing the salience of specific items of information or adding alarms does not seem promising on its own from our study. The exception is VTS where operators must search for specific ships during radio calls. If attention support shall be given, it must account for the framing issues.

The results have strengthened our assumption that domain comparisons—such as our study—are important to perform. For example, from a decision support design standpoint, finding potential commonalities and differences among diverse control domains provides properties and boundaries of the general design space for control applications. Moreover, generating such knowledge can drive the design of AI support towards solutions for typical operator troubles that can be applied across domains, as well as identifying when domain-specific solutions need to be developed. General solutions—applied when appropriate—in terms of software, will also foster a broader use of AI support for operators as well as driving development costs down.

In summary, the temporal JCF analysis showed that the episodes have a set of important similarities across domains. It also showed the importance of understanding the framing of processes-in-focus and the relation between the different control processes. This holds even though the activities—in the domains—show differences. TTM has a sharp difference between pro-active work through the control and reactive systems with verbal communication (versus ATC and VTS). VTS is advisory (compared to TTM and ATC that are control systems). Despite this, there are aspects of the control processes that are analysed here that are rather similar from a JCF perspective.

Finally, our approach to model control processes demonstrates how different domains, sharing common challenges in interaction design for automation support, can be compared. Furthermore, it visualizes how the humans and the automation’s work processes relate, which can support the design of interaction between future, AI-based, automation, and human operators in any domain sharing similar issues.