1 Introduction

Resilience engineering studies, among other aspects, the ability of a socio-technical system (STS) to reorganize and adapt to the unexpected and unforeseen (Hollnagel et al. 2006). Hollnagel (2009) theorizes that a resilient STS needs four essential capabilities: responding, monitoring, learning and anticipating. These capabilities differ in moment and scope—actual, critical, factual and potential—but have in common the need for explicit relevant knowledge and the ability to apply this knowledge. This knowledge is partly available at the sharp end of the system, for instance with control operators, who are a component of the STS interacting with the system environment. However, this knowledge is frequently implicit due to the fact that goal-relevant constraints, required to deal with unforeseen disturbances, are not visible for operators, thus hampering efficient knowledge-based behaviour (Burns and Hajdukiewicz 2004; Rasmussen 1983, 1985; Vicente and Rasmussen 1992). The question arises which method should be used to make the available knowledge explicit. We have addressed this question in the context of a naturalistic observation study conducted at a rail control post. A rail STS is an example of a system that constantly needs to adapt to disturbances, and rail signallers working at different control posts are responsible for making the rail infrastructure available in a safe and timely manner, in the face of daily disturbances.

We propose team reflection (Ellis et al. 2014; Reymen 2003; Schippers et al. 2007, 2014; West 2000; Wiedow and Konradt 2010) as a mechanism for the team at the sharp end to make the resilience-related knowledge explicit. The resilience-related knowledge is defined as the knowledge required to adapt to goal-relevant constraints (Rasmussen1985) imposed by the goals of the STS as they occur due to unexpected and unforeseen events (Branlat and Woods 2010). Team reflection includes behaviours such as questioning, analysis, making use of knowledge explicitly, reviewing past events with self-awareness and coming to terms over time with a new awareness (West 2000). Team reflection, in a loop with planning and action, is commonly used in a broader reflexive process (Schippers et al. 2014; West 2000) where team members collectively reflect upon the team’s objectives, strategies (e.g. decision-making) and processes (e.g. communication). The results of such a reflection can be fed back into the planning and action/adaptation loop to improve team performance (Schippers et al. 2014). However, in our case, the objectives of reflection are to transform implicit to explicit knowledge, at the sharp end, relevant to the resilience of a socio-technical system as a whole rather than the team itself as a focal point for reflection. This knowledge goes beyond the direct responsibility of the team. Implicit knowledge is tacit knowledge, a form of private knowledge that is treated as “informal”, and even, in a sense, “unconscious” knowledge (Day 2005; Polanyi1969), that can be transformed to explicit knowledge (Frappaolo 2008). We are interested in knowledge, relevant to system resilience, acquired throughout the regular work of the rail signallers. Resilience is about the behaviour of the socio-technical system (STS) when it approaches and possibly crosses its boundaries (Siegel and Schraagen 2014a; Woods 2006b). The behaviour of an STS is an interrelated process of all the different participants and technology (Simon 1996; Waterson et al. 2015), which goes beyond the direct responsibility and the team’s span of control at the sharp end. However, the team at the sharp end is exposed and aware outside its span of control and is able to reveal knowledge related to resilience of the whole STS. Therefore, reflection should be applied to system goals rather than team goals.

Rail STSs have three main system boundaries: safety, performance and workload (Siegel and Schraagen 2014c). Since resilience of an STS manifests itself through its ability to adapt and reorganize (Woods 2006b) around its boundaries, it is the case here as well for the rail STS. The movements of the operating system towards these boundaries during a shift of the control room operators, relative to the movements in a previous period, may include hints of system behaviour around the boundaries. As system behaviour around the boundaries is the essence of resilience, we expect that reflection on these movements will cause resilience-related topics to arise. We assume that the broad nature of system boundaries will cause discussion beyond the scope of the team. For example, the team is responsible for setting timely train paths, but through communication, it has knowledge about the personnel on the train, which is beyond its responsibility but can be discussed. These system movements, relative to a previous period, represent weak signals that could possibly signal a “drift into failure” (Dekker 2011). These changes can function as cases to learn from, when the system succeeds in dealing with the situation and things go right as opposed to wrong (Hollnagel 2011). The challenge is to make weak signals explicit and institute a process within the rail traffic control organization to explicitly reflect upon these weak signals to be able to learn from and therefore possibly anticipate subsequent disturbances.

The proposed team-reflection process is applicable, among others, to control rooms of STSs and in particular a rail control post. Our research question in this context and based upon the introduction above is: Does team reflection, on STS movements towards its boundaries, make resilience-related knowledge explicit? This research question can be divided into three sub-questions: (1) What is required, in terms of information presentation to operators, to make knowledge explicit? (2) Is the knowledge that is made explicit resilience-related? (3) Does ongoing practice of team reflection increase the use of resilience-related knowledge? We attempted to answer these questions by conducting an empirical observational study at a rail control post following our proposed process of reflecting with the whole team at the end of their shift. To facilitate team reflection, we developed a prototype, named Resiliencer-performance, which presents movements towards the performance boundary and provides simple analysis functions to retrieve the data behind the daily movements. In order to motivate team reflection on topics related to resilience, we captured movements of the operating system (OS) towards and from the boundaries as described in our previous research (De Regt et al. 2016; Siegel and Schraagen 2014a, c). In this paper, we focus only on the performance boundary and particularly on punctuality. We assume that one boundary is sufficient to study the proposed team-reflection process and trigger discussions that will make resilience-related knowledge explicit. Resilience is a result of interrelated forces and trade-offs caused by the three boundaries (Amalberti 2001; Cook and Rasmussen 2005; Hoffman and Woods 2011). However, movements of the operating state to only one boundary can already be seen as a weak resilience signal (Siegel and Schraagen 2014c). We have focused on the interaction with all three boundaries in a separate publication (Siegel and Schraagen 2017), which enlarges the discussion topics with interrelated cases. In the next Sect. 2, we describe the methods used for the after-shift team reflection, for the Resiliencer-performance implementation and for the analysis. In Sect. 3, we describe the observational study design, and in Sect. 4, we present the results followed by a discussion in Sect. 5.

2 Method

We first describe the setting to understand the context of the methods. The socio-technical system we have studied is a Dutch rail post responsible for the area north and west of Amsterdam with about fifty rail stations and a thousand daily train trajectories (see Fig. 1). The work, performed 24/7, is assigned to rail signallers during the day across four workstations and to one regional dispatcher, who is out of scope of this study. The rail signallers must monitor the system planning and execution. During disruptions, they adjust the planning, manually direct the system and follow safety procedures and protocols including communication with train drivers and other personnel. They enter information about every train delay of more than 3 min through a dedicated application, noting the cause of the delay. This is the only place where they systematically capture their knowledge about the system. The rail signallers perform their tasks, transfer the status to the next signaller at the end of their shift and currently go home without any organized discussion about their work. In case of large disruptions, they may be approached for questioning by staff members or their managers, in most cases a few days or more afterwards, but never immediately after an incident or calamity occurred. The team reflection by the rail signallers that we introduced at the end of their shift is a new activity described in the next subsection. The following subsection describes the requirements and prototype of a tool to support this reflection process. The last two subsections describe the reflection analysis method used to identify the global content and the data-framing method to analyse the expression of explicit knowledge.

Fig. 1
figure 1

The rail map north of Amsterdam with stations mentioned in the text

Full size image

2.1 Team reflection of rail signal operators at the end of their shift

The team mentioned in this paper is a group of rail signal operators working together during a shift at a rail control post. We depict the effects of after-shift reflection on the team’s knowledge in Fig. 2. The signal operators interact individually with the rail STS while being part of it as well. Throughout the interaction, they gain individual knowledge on the rail STS, which is partially made explicit through data entry by the signallers themselves into the system. Changes of the rail socio-technical system are expressed by “to/from boundary movements”. Through the proposed reflection process with analysis of the “to/from boundary movements”, they will exchange some of their knowledge causing it to become explicit.

Fig. 2
figure 2

After-shift reflection of the operator team

Full size image

2.2 Tool for presentation and analysis of relative movements towards the performance boundary

In the introduction, we have explained that operators lack visibility of goal-relevant constraints. This visibility, to stimulate and support discussion, can be obtained by the reflecting team through a tool presenting a view with analysis functions on its shift. The view should present movements towards the system boundaries relative to a previous period (De Regt et al. 2016; Siegel and Schraagen 2014c). In this research, we focus only on the performance boundary as an example of the presentation and analysis which are needed for all the system boundaries (i.e. safety and workload), as described in a separate publication (Siegel and Schraagen 2017). Performance in the rail sector is a combination of punctuality and capacity. In the short term, mainly punctuality plays a role since capacity is nearly constant through its year planning and the effect of disturbances on the capacity is presented through punctuality as well. Punctuality of rail operations is well defined as the difference between planned (i.e. according to the latest published timetable) and actual moments of arrival or departure from a specific station (Goverde 2005; Hansen 2010). However, in our case, we deal with many stations in a large area, many trains, different routes and shift periods, which need an extended punctuality definition. We have taken all the delayed trains (equal or more than 3 min) of a shift within the controlled area and calculated their average delay increase between entering and leaving the area (see detailed description in the "Appendix”).

The presentation should be of a real-time nature to capture data until the start of the reflection, and it should have an ecological interface (Siegel and Schraagen 2017), “to reflect the constraints of the work environment in a way that it is perceptually available to the people who use it” (Burns and Hajdukiewicz 2004, p. 1). The changes in punctuality of an area with respect to a previous period are a value well understood by the rail signaller, but it is difficult to translate a system punctuality number of an area to a specific identifiable component. For that reason, the application needs to provide a simple analysis function, which helps to make the link between the high-level punctuality change of the area to the identifiable component. We used the adjective “simple” to emphasize that the analysis function needs to fit an operator, as opposed to an analyst. We built a prototype, referred to as Resiliencer-performance, fulfilling these requirements (Fig. 3). The application used real-time data and presented in live mode the relative punctuality of the whole area. The area was split into 4 main trajectories to have an initial clue which of the trajectories contributes most to the overall result (left side of Fig. 3). The trajectories were divided into passenger and freight trains, since both have a different characteristic concerning time delays. Passenger trains are tightly coupled to the online published timetable, while freight deviates frequently and has a lower punctuality priority. For the analysis mode (right side of Fig. 3), we implemented a search function to locate the contributing train with only 3 button pushes (see an example in the results Sect. 4.1 Fig. 7). On the right-hand side in the analysis mode, the rail signallers can zoom into one of the trajectories. The top window displays average train delays of train series. Choosing one of the train series will result in the middle display with all the specific trains. Choosing one specific train will result in a delay diagram across its trajectory of this train in the actual shift and the average delay of the specific train in the reference period. The graphs expose the specific train behaviour within the shift and expose patterns or deviations of the same train in the reference period. This information with its reasoning beyond the technical data represents the constraints of the performance boundary in terms identifiable by rail signallers. It is the basis for discussion on information beyond the hard technical figures, enabling the related knowledge to be made explicit.

Fig. 3
figure 3

The Resiliencer-performance in live mode (left) and analysis mode (right, enlarged for readability)

Full size image

2.3 Reflection global analysis

We recorded the team-reflection sessions and analysed them. With this approach, we created a global overview of the content throughout the reflection, which is dependent on the events occurring during the shifts. We segmented the discussion topics in the global overview to areas that are essential to resilience behaviour, to illustrate that the content is resilience-related. The aim of the global content analysis is to show the relation of the discussion topics to resilience. In the introduction, we explained our assumption that reflection on system movements to and from system boundaries, with their margins (Rankin et al. 2013), will evoke resilience-related topics. This follows from the definition of resilience as the ability to adapt and anticipate to changes around system boundaries (Woods 2011). In the following, we describe a coding scheme enabling us to decide whether discussion topics are related to resilience or not. The first category in the coding scheme is “adequately dealing with procedures”. Dekker (2003) claims that organizations should invest in their understanding of the gap between procedures and practice and help develop operators’ skill at adapting. He sums up four reasons why procedures do not work, although they are thought to represent the best thought-out and thus the safest way to carry out a job. The first reason is that there exists a mismatch between procedures and practice. Second, the real world has limited resources to follow the procedures. Third, procedures cannot describe complex situations well. Fourth, procedure following can be antithetical to safety. Discussion about the flexibility of procedures (Bourrier and Bieder 2013) in the real world will improve the performance when applying, adjusting or even neglecting them in real situations. For this reason, our first category of interest to look for in the discussions is “adequately dealing with procedures”. The second category is “communication with counterparties” (Baysari et al. 2008; Murphy 2001; Shanahan et al. 2007). Baysari et al. (2009) reviewed 19 rail safety investigation reports in Australia and found, among others, that the communication between driver and signaller was the only recurring error to contribute to 12 analysed railway incidents. In the UK, research has identified types of communication errors involved in railway incident occurrence (Murphy 2001; Shanahan et al. 2007). These findings emphasize the importance to discuss communication as contributing to resilient behaviour. Moreover, in discussions about communicative actions that need to take place to deal with disturbances, the adaptive capacity (Branlat and Woods 2010; Lundberg and Johansson 2015) is reflected and cross-scale interactions (Woods 2006a) are reflected in discussions about the communication that took place across levels in the rail system (e.g. between posts). The adaptive capacity is also reflected in discussions about similar cases, which is the third category—“reference to similar cases”. Discussing similar cases in the same context shares understandings, decisions made and reasoning, which can be adopted by other team members in future situations (Haunschild and Sullivan 2002; Hovden et al. 2011).

We categorized each discussion topic as follows:

  • Procedure mentioned

  • Communication—this is split up to the main parties the rail signaller communicates with

    • Train driver

    • Police

    • Police

    • “Knoco”—name abbreviation of the station-node coordinator

    • Regional dispatcher or another rail signaller

    • Another person

  • Reference to similar cases

A combination of the three categories, when a train with a deviation from the plan is discussed, indicates a relation to resilience as discussed above.

2.4 Data-framing

We view team reflection on relative system movements as a macrocognitive process of collaborative sensemaking (Fiore et al. 2010; Malakis and Kontogiannis 2014) to explain how knowledge is made explicit. The data-frame theory of sensemaking (Klein et al. 2006; Rankin et al. 2016) postulates that elements are explained when they are fitted into a structure that links them to other elements. The term frame is used to denote an explanatory structure that defines entities by describing their relationship to other entities (Klein et al. 2007, p. 118). The initial trigger for the sensemaking process in our domain of interest is information (data) on relative operating state movements towards the boundary. Each of the reflecting team members is exposed to these data and frames it based upon his or her own implicit knowledge. In Fig. 4, we have depicted the interaction between the individual data-framing and the shared data-framing of a team reflecting together. As seen in the data-frame diagram in the left side of Fig. 4, the individual is questioning his data-frame model by tracking anomalies, detecting inconsistencies and judging plausibility. The questioning may cause the need to reframe or to elaborate/preserve the data-frame. When some equilibrium is achieved, one of the team members may express himself explicitly and share his data-frame with the team. In the social domain, the team will verbally go through a similar process of discussing the shared data-frame (the right side of Fig. 4) by questioning and when needed by reframing or elaborating. During the discussion, the knowledge is shared and becomes explicit and thus available for all team members. The data-frame cycle in the team domain will influence the individual domain, through personal questioning and new frames to compare with. This interaction between the team and individual is continuous, until a satisfactory equilibrium is found, through which related explicit knowledge is made available. This generic process of an individual making his implicit knowledge explicit through a social team process is not new. However, we are trying to explain how resilience-related knowledge is made explicit. We use the behaviour of the system towards its boundaries as stimulator, being a derivative of resilience by its definition (Woods 2006b), as mentioned above. The relative system movements also present drifts towards the boundary, which may not be noticed on a daily basis, but may be amplified by presentation of work-shift data in comparison with previous longer periods. Our research focuses on making drifts towards the boundary more notable for team members, such that they can explicitly reflect upon this drift during team reflection and collaborative sensemaking. This noticed drift is seen by Dekker (2011) as a resilience component to act upon. The data-frame theory depicts the process from relative system movement data, through relevant framing, towards resilience-related knowledge.

Fig. 4
figure 4

Individual Data-Frame interacting with shared Data-Frame

Full size image

Within the data-frame theory of sensemaking (Klein et al. 2006), a process of questioning, elaboration or reframing is described. In our method of data-framing, we are mimicking this process to show how knowledge is made explicit. We record the verbal expressions during the discussion and transcribe them in line with verbal analysis (Chi 1997). This time we take a few sentences expressed on a topic and refer to it as data, which needs a frame to fit in. The frame, a construct to that of cognitive schema (DiMaggio 1977), is an organized pattern of thought or behaviour that organizes categories of information and the relationships among them. Since the reflection starts with the Resiliencer-performance, the first frame discussed is triggered from that domain. The main frame triggered will be the Delay frame, triggering thought and information on the occurrence of delays, but other frames can be triggered as well, for example, a Graph-frame concerning information of graphs, or a Reference-frame about thoughts and experiences of the reference period. This method is used to describe and cluster the knowledge made explicit, where the mapping within a frame group is only a means to structure the knowledge. In the results section, we will present a case which on the one hand clarifies the method and on the other hand shows, with help of sequenced data-framing, how knowledge is made explicit and how the Resiliencer-performance has triggered the discussion, through the initial frame.

3 Observational study design

The study design at the Dutch rail post described above introduced team reflection with consent of the management and the rail signallers at the rail control post (Fig. 6, in the top centre). At the end of the rail signal operators’ duty (Fig. 6, in the bottom centre), the team discussed delays within its controlled area. For that, they used the Resiliencer-performance (Fig. 6, left side). The application was configured for the specific rail post under investigation. It presented in live mode the punctuality status and provided in analysis mode the ability to search for logistic details (i.e. the delay progress of a specific train). The post-area was split up into four main trajectories covering all stations, and each trajectory was controlled by two out of the four workstations. This caused at least two rail signallers to relate to the results of a main trajectory. The results of the four trajectories were joined into a result of the whole post during a shift.

The observational study took first place on one tryout day with the new developed Resiliencer-performance prototype, which was successful. A full working week followed from Monday to Friday all with two shifts, except for Friday when only the morning shift was taken into account. The early shift lasted from 6:30 AM until 2:30 PM and the late shift from 2:30 PM until 10:30 PM. The reflections took place at 2 PM for the early shift and at 9 PM for the late shift for about 30 min (see Fig. 5 for an impression).

Fig. 5
figure 5

Team reflection with the Resiliencer-performance

Full size image

The four rail signallers on duty voluntarily finished their work between half an hour and an hour before their scheduled ending time, during the observational study period, for a reflection session together with their team leader (the next team voluntarily started earlier to fill in this gap). They asked themselves the following generic questions:

  • Did our shift today proceed better than the average of last period? Why?

    • What were the circumstances for the difference?

  • Which of the identified circumstances could occur again in the future?

    • What can we learn from that?

  • How can we deal with these circumstances and what can we do differently?

For answering these questions, they used the Resiliencer-performance and analysed the numerical punctuality progress in their area. However, reasoning beyond the numerical data could only be accomplished with help of their personal knowledge and notes made during their shift (Fig. 6, in the centre). We recorded the discussion and analysed it (Fig. 6, top right side) as described in the previous section. The protocol guiding the observations included an oral recorder consent, due to cultural constraints and specific request of the post management, and was approved by the Ethics Committee of the University of Twente (No. BCE15199 dated 17-4-2015).

Fig. 6
figure 6

Illustration of study design

Full size image

4 Results

4.1 Sequenced data-framing case: suicide attempt (human approaches the rail)

We describe here a case, which we use to explain how knowledge was made explicit, throughout a reflection session, with help of the data-framing model. The case is a suicide attempt, which occurs almost daily within the Netherlands. In 2011, 205 attempts and, in 2012, 188 attempts were successful (CBS—Statistics Netherlands 2015). This frequency causes adaptations in behaviour of train drivers when dealing with human approaches to the rail tracks. The standard procedure of a rail signaller in this case is as follows. When a train driver reports the rail signaller that he has seen a person approach the tracks, the rail signaller alerts other train drivers on the trajectory to slow down and watch for that person. Only when a second time the person has been spotted, the rail signaller will call the police. The train drivers are expected not to stop unless the person is on the tracks and should let the police approach the person in question. However, reality does not follow these stated procedures. During our observation period, the situation of a person approaching the rail occurred twice, without deadly ending. The first time train drivers stopped near the person until the police came. The second time the train driver decided on his own to take the person into his cabin up to the end station where he delivered the person to the police. These cases arose during the reflections. We have used the data-framing theory to show the process by which information is made explicit and how it is related to the initial trigger of the Resiliencer-performance. In the left side of Fig. 7, we marked on the Resiliencer-performance in analyse mode the steps followed. The signallers identified that the maximum delay on the trajectory Den-Helder–Zaandam was 13 min (marking 1a) and searched to find the train with that delay on the trajectory (marking 1b). Marking 2 is the identification of the 3000 series, and marking 3 is the identification of the specific 3023 train. The delay trajectory is shown in the results window (marked R) and enlarged at the right side of Fig. 7.

Fig. 7
figure 7

The suicide case (Resiliencer-performance in analyse mode)

Full size image

In Table 1, we have detailed the verbalization of the reflection, the related frame, the frame group it belongs to and the explicit knowledge within the team-reflection domain. The first step is associated with the Resiliencer frame group, making sense of what is presented on the screen. The next step is the identification of the actor, and further framing is used to get to the relevant details of the case discussed. Table 1 includes abbreviations of the following persons: team leader (TL) and rail signaller (RS).

Table 1 Data framing making available explicit knowledge
Full size table

To explain how knowledge was being made explicit, team reflection can be seen as a frame sequence process, after which each frame has been elaborated or reframed. The first frame is from the Resiliencer frame group concerning the relative system movement—the performance of the shift relative to a previous period, the performance of the main trajectories and extreme performance. Reframing causes focus on the actors, train 3023 with its train driver, and is followed by focus on the case and details within. The following frame sequence occurred in our case:

  • Delay frame was triggered by the Resiliencers relative system movement

    • an extreme delay (13 min) on trajectory Den-Helder–Zaandam

  • Train frame, as an actor, was triggered by the Delay frame

    • Series 3000 with train 3023

    • Train 3023—Driving from Den-Helder to Amsterdam Sloterdijk. 0 min delay at Alkmaar (06:42) and 11 min delay at Heiloo (06:46 + 00:11).

  • Suicide frame was the case frame triggered by the Delay and Train frames

    • Reporting of a person approaching the rail

  • Procedure (formal versus reality) frame was a detail frame triggered by the suicide frame

    • Reporting of and to the train drivers

    • The stop of the train driver not according to the procedure

    • Point of interest

      • The train driver. His behaviour when locating the person near the rail.

    • Higher abstraction

      • Train drivers and suicides

Through the sequence of data-framing, rail signallers revealed to each other their work “as done” (Cowley and Borys 2014, p. 21; Lundberg et al. 2009, p. 1298) as opposed to the work “as designed” or “as described” in the procedures they need to follow. In our case, they learned from each other how train drivers behave in reality when a person is approaching the rail, being triggered by information of the Resiliencer-performance. The standard procedure of ordering the train driver to continue his journey, to wait for a second spotting and then to call the police, did not work. Understanding the real world, as it unfolds, includes knowledge, which may be important in next occurrences of similar cases and can also help in new cases. When the unforeseen and unexpected occur, it may be crucial to use knowledge and experience on accepted deviations from standard procedures.

4.1.1 Drift towards the boundary and the interdependent relationship

Another aspect seen in this case is the slow drift towards the boundary. The average delay of delayed trains during this shift was 1.8 min, while the delay in the previous month was 1.3 min. This means that the movement towards the performance boundary was only 30 s, where only trains with 3 min or more are counted officially. If the prototype would not have presented data for discussion on this seemingly small movement, then in-depth discussion, making related knowledge explicit, may not have taken place on this suicide case. Moreover, when searching for the reason of this small daily drift, it appeared that the same train (no 3023) had 6 delay occurrences in the last month (see the brown reference line in Fig. 8). All delays started between Alkmaar and Heiloo, indicating that this area causes more delays due to suicidal behaviour. Curiously, this particular fact has been discussed in the second chamber of the Dutch parliament as well (Minister van Verkeer and Waterstaat 2010), an example of the interrelatedness of different areas influencing the events and behaviour of the rail STS. These facts can be used for learning and anticipating. But beyond these interesting understandings, it becomes clear that the relationships of all entities are very interdependent (Woods and Branlat 2010). It is not a matter of a direct link between a signal and its cause. The signal hides many interrelated entities, which can be revealed through the knowledge exposed by the sharp end. Like in our case, the following short and incomplete list was revealed: train-delay, suicide, deviation from procedures, second chamber, many occurrences between Alkmaar and Heiloo, etc. The reflection on this weak signal, a delay drift, caused discussion and awareness of complex interrelationships and enhanced the explicit knowledge as an important resilience behaviour component (Dekker 2011).

Fig. 8
figure 8

Change of procedure and communication ratio with respect to number of deviancy trains

Full size image

The description above is a typical example of a weak resilience signal (WRS) triggering discussion beyond the movement towards the boundary. It supports our assumption that even a WRS on one boundary can reveal knowledge with a variety of dimensions and interrelationships. This case with only 30 s drift got attention because on that day, train 3023 had the largest delay, which was the reason for the team to choose that train in the sequence of Sect. 4.1. The team could choose any deviation, which draw their attentions. The triggers, weak resilience signals, make knowledge explicit which include resilience-related components, analysed for the observation week in the next Sect. 4.2.

4.2 Global analysis

The global results of the reflections are presented in Table 2. The first column contains the date, the day of the month (April 2015, last day was on the first of May) and the shift (E = early and L = late). The second column contains the number of trains discussed throughout the reflection. Some trains were discussed, without having any issue, having a role in the logistic stories of the rail signallers. The next column contains the number of train cases having an issue. The column after that contains the number of procedures discussed. In hindsight, it became apparent that any train having an issue was reasoned about with the help of at least one procedure to explain the actions. These procedures were central in the team’s reflection, and signallers discussed whether they were or should be obeyed as well as the need to adjust them. Some procedures concerned logistics decisions, while others consisted of sequences to act. An example of a logistics decision is the priority of an express-train over a stop-train. An express-train has priority over a stop-train until it is delayed more than “x” minutes, minimizing extra delays within the system. An example of a sequence to act procedure is the moment to call the police when the train driver has spotted a person near the rail tracks. The standard procedure is to call all train drivers on the trajectory to spot the person for a second time to make sure the police will not come unnecessarily. However, this second spotting may cause a delay being too late to prevent an accident. The next column contains the number of communicators the rail signaller has contact with, as mentioned in the reflection. The first sub-column lists the total of train driver, police, colleague who may be another signaller or a regional dispatcher, or any other person. In most cases, the train driver is contacted, while the others are strongly dependent on the situation. The last column contains the number of reference cases cited during the reflection. Most cases where procedures were mentioned were tied to another reference case.

Table 2 Global analysis results
Full size table

The test week was very quiet, yet cases occurred and were discussed. On average, 7.2 trains were discussed during each shift. On average, 3.1 trains out of the 7.2 were considered to have an issue.

From the information in Table 2, we can test whether there was an increase in the use of resilience-related knowledge categories throughout the observation week. Assuming that the number of procedures and communicators discussed is an indicator of the quantity, we can divide it by the number of delayed trains to test an increase per delayed train. The results are plotted in Fig. 8, showing a shallow increasing trend line for both ratios during the week. Days 29L and 30L were not plotted since no deviated trains occurred on that day. Days 9E and 9L were test days of the Resiliencer-performance with full compliance of the reflection protocol, justifying to add the data to the observation week. Those days were actually the first time the reflection was performed in operations. The increase during the observation was not substantial, although a jump on the procedure rate occurred on day 28L. This result indicates a possible increase in the resilience-related knowledge categories discussed over the course of the observation week. A longer period of observation is needed to strengthen our hypothesis that this type of knowledge increases when team reflection is facilitated by presenting goal-relevant constraints.

5 Discussion

The purpose of this paper was to test the proposition that team reflection on system movements towards it boundaries will make resilience-related knowledge explicit. The high-level requirements of a tool used by the operators during their team reflection are: (1) the interface should be ecologically designed (Burns and Hajdukiewicz 2004) combining all boundaries; (2) the data must be of a real-time nature; (3) the interface includes simple analysis functions to relate system boundary values to specific identifiable components. These requirements give an answer on the first question posed in the introduction on the needs of the operators to make knowledge explicit. The tool was prototyped and used during our observation at a rail control post. Through the analysis of their discussions during a specific case, we showed how knowledge was made explicit, using the data-frame theory of sensemaking (Klein et al. 2006). In this case, we were able to show as well the interrelationships between entities during a drift towards the boundary. With help of a global analysis, we showed that the explicit knowledge is related to resilience and that its use indicates a possible increase throughout the observation. These findings give an answer on the second and third question in the introduction on the relation to resilience and the increase in knowledge.

However, proving that a specific knowledge detail will play a role in resilient behaviour when the unexpected or unforeseen occurs is extremely difficult. Other researchers (Heese et al. 2014; Herrera et al. 2015; Van der Beek and Schraagen 2015; Woods et al. 2014) have looked at other properties than knowledge to assess the resilience of an STS. They strongly based their research on two fundamental theories: the four cornerstones of Hollnagel (2009) and the stress–strain theory of Woods and Wreathall (2008). Schraagen (2015) has introduced the relation between networks, knowledge and resilience. We have made a first attempt to show the relation between knowledge and resilience in an empirical setting through reflection (Ellis et al. 2014) and the data-frame theory of sensemaking (Klein et al. 2007). We showed the relation indirectly by splitting the analysis of the reflection discussion into two—global and specific. In the global analysis, we showed the relation of the topics to resilience through three categories derived from previous research—“adequately dealing with procedures”, “communication with counterparties” and “reference to similar cases”. In the specific analysis, we showed, with help of sequenced data-framing, how knowledge was made explicit in those resilience-related topics. In addition, we demonstrated the relation of the frames to the Resiliencer-performance, a support tool that presents relative system movements towards the performance boundary, where resilience behaviour is needed. The results support the assumption that knowledge is related to resilience, but a further quasi-experiment would definitely strengthen our argument. In addition, we tested the increase in the use of resilience-related knowledge throughout the observation week. To this end, we controlled for the number of delayed trains, as any delayed train would surely lead to more knowledge being made explicit. By the end of the week, an increase was observed in resilience-related knowledge being discussed, even when controlling for the number of delayed trains, indicating that the introduction of team reflection was successful. Of course, this result needs to be interpreted with caution as we did not observe over a longer period of time and cannot state whether this effect would hold up.

The observation was done in real operations without a control group. We did not organize reflection sessions without the Resiliencer-performance, since in real operations, the cases vary largely from day to day and would be difficult to compare. However, rail signallers stated that, in the past, previous debriefing attempts had been less successful than their current team reflection sessions, because:

  • The discussions were not interesting and focused mainly on major events during the shift, which all of them were already aware of;

  • No online system exists today, which gives a good picture of the shift. The operational systems are real time and are not designed for debriefing.

The reflection tool we introduced addressed the shortcomings of previous debriefing attempts by being an online tool dedicated to the after-shift team reflection as it presents the information from a different perspectives than the major events during the shift. Although the field observations took place during a quiet period, the tool was able to trigger topics rail signallers were not directly aware of. An example is the identification of delay trends compared to previous periods. Moreover, the tool caused discussion on things that go right and is therefore in line with resilience thinking (Hollnagel 2011). Rail signallers became explicitly aware of procedures not being followed and shared this knowledge. We assume that through this awareness, learning took place and that in subsequent scenarios, anticipation and responding might be improved. As these activities are thought to underlie resilience performance (Hollnagel 2009), we may assume resilience of the team as a whole improved. However, due to limited observation possibilities, we were in the current study unable to look at the team’s behaviour in the long run. Therefore, any conclusions on the causal relation between knowledge and resilience are premature at this stage.

We have shown in this paper how the resilience engineering approach of sensemaking on post-event reconstruction adds value beyond the simple traditional system monitoring. However, the approach contains an inherent complexity. The success of the reflection making resilience relevant knowledge explicit depends on the individuals, group dynamics and culture of the environment (Gabelica et al. 2014; Schippers et al. 2014). Moreover, it also depends on the information provided to the team to reflect on. It is in the end the group’s responsibility to identify the right event and reveal the relevant information to each other. More research is needed to understand ways to overcome these limitations and provide methodologies that result in a consistent set of information under similar conditions.

The team reflection in the experiment focused only on the performance boundary, while in theory (Siegel and Schraagen 2014b), there are two more boundaries: the safety boundary and the workload boundary. The concept of gaining knowledge of all the complex interdependencies beyond this boundary was discussed above in the context of the suicide case study. A more complete picture may arise through the usage of the three boundaries. An experiment with all three boundaries (Siegel and Schraagen 2017) will give more empirical insight on the contribution of team reflection in a broader sense and will deepen the understanding of collaborative sensemaking on related subjects from different boundaries of a system.