1 Introduction

Conducting incident analysis in healthcare systems is crucial for promoting and improving patient safety (Johnson 2004). This requires identifying and reducing risks through the development and application of safety recommendations or preventive measures to prevent their re-occurrence (Salmon et al. 2012). However, in the current practice of incident investigations at the National Health Service (NHS), there has been a dependence on the use of Root Cause Analytical (RCA) techniques to determine contributing factors relating to the accident that occurred (Canham et al. 2018). While the concept of “systems thinking” has become a dominant paradigm relating to understanding why adverse events occurred, approaches incorporating this paradigm have not been extensively applied for incident investigations in healthcare systems. This is has led to the concept coined by Underwood and Waterson (2013) regarding the “research-practice” gap relating to the benefits of adopting this thinking not being practically implemented in clinical practice.

The AcciMap approach is considered an example of systemic approaches (Salmon et al. 2010; Salmon et al. 2012) which was developed for graphically depicting multi-causal chains consisting of events, decisions and contributing factors (Svedung and Rasmussen 2002; Rasmussen and Svedung 2000). While the AcciMap approach in addition to other systemic approaches like STAMP (Systems Theoretic Accident Modelling Process) (Leveson 2002; Leveson et al. 2004; Qureshi 2008) and FRAM (Functional Resonance Accident Method) (Hollnagel 2004) provide different perspectives to how accidents are analysed, there has not been a prevalent adoption of these approaches despite their benefits (Underwood and Waterson 2013). This version was developed by Branford (2007) based on existing versions of the original AcciMap (Rasmussen and Svedung 2000; Vicente and Christoffersen 2006) as a way of providing a consistent methodology for users in determining why the adverse event occurred (Branford 2011).

This paper discusses patient safety practitioners' perception of the application of a formalised version of the AcciMap approach (method) based on their findings using the model usage characteristic framework (Underwood and Waterson 2014). This framework focused on important characteristics of an accident analytical approach including usability (ease of use), validity, reliability, and graphical representation of the accident (Underwood and Waterson 2014). It was initially utilised in a field study for evaluating practitioner’s perception of the application of the STAMP model (Underwood et al. 2016) and this is applicable in gaining insight on their first-time application of the AcciMap approach. While there have been a few studies of the application of the AcciMap approach in the health domain (Waterson 2009; Woo and Vicente 2003; Vicente and Christoffersen 2006), there has not been any significant report of its application in NHS clinical practices. Although it is important to state that since this study, there has been a growing application of systemic approaches including the AcciMap approach in NHS practices. It is also being utilised with other systemic approaches at the National level most notably the Health Safety Investigation Branch (HSIB), a branch of the NHS recently established in 2017.

2 Rasmussen’ risk management framework (RMF)

Before briefly outlining the AcciMap approach, it is necessary to establish the broader framework; the Risk Management Framework where it was derived from. The Risk Management Framework (RMF) developed by Rasmussen (1997) underpinned the notion of safety as an emergent characteristic of complex sociotechnical systems. RMF is also “a prominent systems-theory based model for describing work systems composed of various labels, and argues that safety is impacted by the decisions and actions across all levels (e.g. politicians, chief executives, managers, supervisors), not just by those of front line operators alone” (see Fig. 1) (Donovan et al. 2015; Salmon et al. 2012; Waterson and Jenkins. 2011).

Fig. 1
figure 1

Socio-technical model of system operations (Svedung and Rasmussen 2002)

Full size image

Each level of the Risk Management framework is connected by a flow of information in a top–down approach from the external level to the frontlines (physical level) (Svedung and Rasmussen 2002). This flow of information from the top denotes decisions taken by different external entities and then the state of the system is taken upwards helping to “inform decision making and action at higher levels” (Donovan et al. 2015). Interactions within and between these system levels also control the performance and safety of the system (Trotter et al. 2014; Waterson and Jenkins 2010). While the AcciMap approach is underpinned by this framework (Salmon et al. 2010), additional tools also include Actor Maps, Conflict Maps and Info flow Maps (Waterson and Jenkins 2010).

3 The formalised AcciMap approach

AcciMap (Accident Mapping) is a retrospective accident analytical approach used for representing and analysing systemic failures (Rasmussen 1997). It can be utilised as either a standalone method or as a part of a broader Risk Management Framework (RMF) for the development of risk management strategies (Waterson and Jenkins 2010). This approach also describes or depicts decisions, actions or events within and between six levels consisting of Environment, Actor Activities, Management, Company, Regulations/Associations, and Government (Salmon et al. 2012; Hopkins 2000). Also, it allows analysts to graphically represent causal relationships (cause and effect) between identified causal/contributing factors within and between levels that led to the adverse outcome(s) (Branford 2007; Branford et al. 2009). As highlighted, the formalised version of the AcciMap was developed based on the original AcciMap template presenting a more simplified approach as depicted in Fig. 2.

Fig. 2
figure 2

(Adapted from Branford 2007; Branford et al. 2009)

Standardized AcciMap structure

Full size image

The main difference between the original version and the formalised representation of the AcciMap approach is that the former has six (6) levels of abstraction while the latter was reformatted into four (4) levels. The latter approach did not include “Equipment and Surrounding level” and both “Technical and operational management” and “company management & local area government” was merged as “organisational”. The external level of Branford’s approach includes the merging of “Regulatory bodies & Associations” and “Government policies”.

The AcciMap Methodology involves several steps in the analysis of both incidents and accidents in determining both proximal as well as system-wide failures. Several steps from the guidelines can be repeated until the final AcciMap outcome is satisfactorily obtained. These procedures involve all the six levels in the AcciMap structure (Salmon et al. 2011):

  1. (1)

    Collection of data relating to the accident/incident which involves activities, such as interviews, analysis of reports and observation recordings.

  2. (2)

    Identifying failures relating to the accident from all the six levels; equipment and surroundings, actors and physical processes, technical and operational management, local area government/company management, regulatory bodies and associations and government policy and budgeting.

  3. (3)

    Identifying factors relating to the accident in step 2 and as well as factors relating to each of the levels.

  4. (4)

    Identifying failures at other levels to determine the causal factors and consequences. These can be added to a “rough” diagram.

  5. (5)

    Construction of the AcciMap diagram.

  6. (6)

    Finalizing and reviewing the constructed AcciMap diagram which may require several iterations to have a completed output.

The processes involved in analysing incidents using the formalised AcciMap approach (Branford et al. 2009) according to the guidelines developed by Branford are essentially similar to the above steps regarding the general AcciMap construction include the following:

  1. (1)

    Creating a blank AcciMap format on which to arrange the causes/contributing factors

  2. (2)

    Identifying the adverse outcome of the incident

  3. (3)

    Identifying contributing factors based on the incident report

  4. (4)

    Identifying the appropriate AcciMap level for each contributing factor identified

  5. (5)

    Preparing the contributing factors representative of each AcciMap level

  6. (6)

    Inserting causal links (relationships) to depict cause and effect between contributing factors

  7. (7)

    Filling in the gaps left in the causal chains where information is missing

  8. (8)

    Checking the causal logic and making sense of the sequence of events

  9. (9)

    Formulating safety recommendations that are practical and feasible

4 Why the AcciMap approach?

Root Cause Analytical (RCA) techniques have been used as toolkits for investigating and analysing serious events in healthcare (Johnson 2004; Qureshi 2008). However, they have been noted to assume a linear approach regarding cause and effect to accident causation and so unsuitable when analysing incidents from a sociotechnical perspective (Canham et al. 2018; Qureshi 2008). These techniques are mainly focused on analysing errors at the front end of the spectrum and not considering in-depth system weaknesses and why they occurred. The AcciMap approach provides a means of analysing accidents in complex socio-technical systems and is cited as one of the most popular systemic approaches in the research domain (Underwood and Waterson 2013).

In terms of its application, the AcciMap approach is in a way similar to the use of the Five (5) Why technique when determining what happened. However, the AcciMap approach depicts multi-causal (cause and effect) linkages between causal/contributing factors (Branford et al. 2009). There has been a continued dependence on RCA techniques which do not incorporate the benefits of applying systems thinking in analysing accidents, why they occurred and preventing their re-occurrence. The AcciMap approach provides this benefit of analysing existing weaknesses in systems as complex and socio-technical as healthcare which will enable users to develop safety recommendations to address these weaknesses.

5 Study aims

This paper focuses on patient safety practitioners’ perception of the use of Branford’s formalised AcciMap approach for incident/accident analysis. To achieve this, a formal workshop with participants across NHS boards was organised by the Healthcare Improvement Scotland (HIS). The objectives of this study include the following:

  1. (1)

    Obtaining participants’ perception of the application of the formalised AcciMap approach in the analysis of the “Wrong Patient” case incident (Chassin and Becher 2002).

  2. (2)

    A qualitative evaluation of the AcciMap approach based on the usage characteristics criteria (Underwood and Waterson 2014).

  3. (3)

    Exploring if there are benefits in using this approach as part of adverse event analysis for health and social care events

6 Research methodology

This study will involve a case study incident relating to “Wrong Patient” obtained from the Annals of Medicine journal (Chassin and Becher 2002). The incident was selected based on the nature of the type of error that occurred and noted by the authors. Using a qualitative case study approach as noted by Yin (1994), participants will then apply the formalised AcciMap approach to determine contributing factors identified and safety recommendations proposed. Participants were introduced and trained in the use of the AcciMap approach within the first hour. Participants were given two hours for the AcciMap analyses exercise. The following sections outline the study methodology.

6.1 Participants

A total of 15 participants took part in the AcciMap training workshop. Information and consent forms were given to the participants and filled out before the workshop. Participants who accepted the invitation composed of Eight (8) territorial (regional) NHS boards (out of a total of 14 across Scotland) and three (3) special NHS boards. Roles and responsibilities of the participants across different these NHS boards in Scotland included Clinical governance, Risk management, and Health and Safety management (see Table 1). All participants were experienced with the use of Root Cause Analysis (RCA) techniques, fishbone diagrams for incident analysis in their respective boards. Permission to use the approach and its guidelines were granted (Branford 2016) and the ethics approval for the workshop was obtained from the Healthcare Improvement Scotland (HIS).

Table 1 List of participants involved in the AcciMap workshop
Full size table

6.2 Training/materials provided

Materials including the case study information and the AcciMap guidelines originally developed by Branford were distributed to the participants prior through the Healthcare Improvement Scotland (HIS). The materials were given to them prior to the commencement of the workshop through the HIS Senior Lead to enable them to familiarise themselves with the case study. On the day of the workshop, participants were introduced to the AcciMap approach as part of the broader Risk Management Framework (RMF). Rasmussen’s theory and the socio-technical structure describing the AcciMap approach was also highlighted and methodology of application within the first hour of the workshop. A case example of the application of the AcciMap approach was also described to the participants.

6.3 Procedures

During the first section of the training, the foundation of the AcciMap approach was explained to the participants as well as a graphical representation of its application on an example incident. Participants were then subsequently divided into three groups of five each; teams A, B and C. The case study information given to them only contained the chronology (timeline) of events without any initial analyses and discussions from the original authors to avoid bias. Each team commenced their analysis of the case incident within the next two hours assigned for the exercise and each of the team’s discussions was separately audio recorded. Large papers and sticky notes were given to the teams to use to construct their AcciMap outcomes and safety recommendations were also developed within the time that was permitted. After their analyses, the teams were then asked to review each other's results before a final discussion. Questionnaires were then distributed to participants after the focus group discussions were completed to end the workshop.

7 Incident summary

The case report highlighted a type of medical error that occurred in a US-based hospital where the wrong patient underwent an invasive procedure (Chassin and Becher 2002; Johnson 2004). This incident was indicated to be very distressing and warranted attention. In addition, this type of error was also under-reported according to Chassin and Becher (2002). The scenario involved a 67-year-old patient who was admitted to the hospital for cerebral angiography but mistakenly underwent an invasive cardiac electrophysiology procedure. A second patient, a 77-year-old patient was transferred from another hospital for a cardiac electrophysiology procedure. Her procedure was delayed for two days and was intended to be the first case on the day of the first patient’s planned discharge from the hospital.

The full timeline of the chronology of events is summarised in Table 2. This incident was reviewed and analysed using the institution’s root cause analysis where several distinct errors were discovered. According to the article, “no singular error” was identified which could have led to the adverse event itself (Chassin and Becher 2002).

Table 2 Summary of events (Timeline)
Full size table

This case incident was selected based on the above reasons regarding the type of error and the location where it occurred. The case also contains a narrative timeline describing the activities that took place which also gives participants the opportunity of not only picturing the incident scenario but applying systems thinking concepts using Branford’s AcciMap guidelines. Discussions relating to the nature of the incident could prove to be insightful as this case was not something that any of the participants had experienced in their respective NHS boards.

8 Data collection and analysis

The data sources from the workshop consisted of audio recordings from each group designated as Team A, B and C as well as survey data on the evaluation of the AcciMap approach. The AcciMap outputs from each group were also collected including safety recommendations.

8.1 AcciMap analysis exercise

After the exercise, participants from each team reviewed and compared their findings with what other teams did in producing their outcomes. AcciMap results are compared and contrasted for similarities and differences in terms of contributing factors using content analysis as a qualitative reliability measurement (Branford 2007). Outcomes are also compared with external analysis of the case incident in terms of if similar contributing factors and safety recommendations were identified (validity assessment).

8.2 AcciMap survey questionnaire

The evaluation questionnaire used for the workshop was adapted from previous fieldwork utilizing another systemic accident model (STAMP) (Underwood et al. 2016). Data collected from the survey were analysed using Microsoft Excel and R, a statistical software. The questionnaire consisted of twenty-two (22) questions relating to important aspects of an accident analytical approach including usability and validity. These aspects were also used to evaluate the STAMP approach in a small investigation study with safety practitioners in the Railway domain. The questionnaire was distributed to the participants after the analysis exercise was completed.

8.3 Audio recordings

Audio recordings collected from each team during their analyses and final group discussions were manually transcribed to determine themes relating to contributing factors identified. During the final group discussion, their experience regarding the application of the AcciMap approach, advantages, limitations, and how it can be further improved as a tool for incident analysis were also recorded and manually transcribed.

9 Findings from the workshop

Findings based on the survey instrument and the respective AcciMap results from the participants’ analysis of the case study are divided into the following sections:

9.1 Analysis of the AcciMap evaluation survey

Thirteen (13) out of fifteen (15) participants completed the survey after their analyses. However, there missing gaps (2) based on questions 13 and 20 filled by two participants. Based on the first three survey questions (see Tables 3 and 4), ten (10) participants were familiar with the concept of “systems thinking” before the workshop (76.9%) but out of the ten (10) participants, only three (3) of them were vaguely familiar with the AcciMap approach.

Table 3 Question_One Were you familiar with systems thinking?
Full size table
Table 4 Question_Two Were you aware of the AcciMap method?
Full size table

Three groups were generated based on questions 1 and 2 as indicated below:

Group 1: Familiar with “systems thinking” and unaware of the AcciMap approach (7 respondents);

Group 2: Unfamiliar with “systems thinking” and unaware of the AcciMap approach (3 respondents);

Group 3: Familiar with “systems thinking” and aware of the AcciMap approach (3 respondents).

The levels of satisfaction regarding the AcciMap approach appear to be different between each group identified (see Fig. 3). In addition, group 1 indicated the best level of satisfaction on the use of the AcciMap approach. An Analysis of Variance (ANOVA) analysis was also performed to gain insight into the differences in satisfaction from three groups regarding the knowledge of systems thinking and their awareness of the AcciMap approach. The results indicated in Table 4 suggests that there is no evidence to reject the null hypothesis and hence no evidence of any differences amongst the groups.

Fig. 3
figure 3

Density of the three groups relating to questions 1 and 2

Full size image

The summary of descriptive statistics based on questions 4 to 22 (see Table 5) where mean and standard deviation values from the survey data are indicated. Minimum and maximum values represent the lowest and highest values on the Likert scale (from “strongly disagree” (0) to “strongly agree” (6)) used to rate each question (see Appendix A). Based on the exploratory analysis of the responses, participants generally considered the AcciMap approach to be a suitable approach for analysing accidents (Question 4). There was also a favourable agreement regarding the method’s ability to represent causal relationships between different levels (Question 8) and as well as promoting collaboration (Question 17).

Table 5 ANOVA for three groups
Full size table

Question 6 focused on the effectiveness of the AcciMap approach in identifying contributing factors based on technical components, human factors, organisational, environmental and external issues (see Tables 6, 7, 8, 9, 10 and 11). However, participants’ responses indicated a considerable amount of neutral responses particularly for question 6a–e.

Table 6 Descriptive statistics (Questions 4 to 22) (Igene et al. 2017)
Full size table
Table 7 AcciMap effectively analyses the contributing factors to an accident from Technical components (Question 6a)
Full size table
Table 8 AcciMap effectively analyses the contributing factors to an accident from Human factor issues (Question 6b)
Full size table
Table 9 AcciMap effectively analyses the contributing factors to an accident from Organisational issues (Question 6c)
Full size table
Table 10 AcciMap effectively analyses the contributing factors to an accident from Environmental issues (Question 6d)
Full size table
Table 11 AcciMap effectively analyses the contributing factors to an accident from External issues (Question 6e)
Full size table

There was a significant amount of neutral responses regarding the questions relating to the effectiveness of the AcciMap approach when analysing contributing factors. Reasons for this could be attributed to their very first attempt at applying the AcciMap approach using the guidelines, understanding of the incident and lack of substantial evidence at both organisational and external levels. For example, one of the factors at the organisational level; “electrophysiology laboratory computer not in sync with the hospital’s main computer system” contributed to the patient subsequently not being properly identified by the medical staff. However, further investigation will need to be carried out to ascertain why that is the case and parties responsible would include the technical/IT department and hospital management. Other organisational factors not based on explicit evidence but could be deduced based on actions of medical staff include how they interacted with the patient (obtaining consent), misidentification of the patient and inadequate communication with other staff (not using the full name of the patient).

There were also mixed responses from participants regarding the application of the AcciMap approach being a time-consuming process and the sufficiency of the training for effective use (Question 21 and 22). Other aspects of the AcciMap approach regarding its ability to identify unsafe decisions (question 14) and terms and concepts being clear (question 12) also indicated mixed responses including positive and neutral responses. This may have been influenced by their level of satisfaction regarding the sufficiency of the AcciMap training (question 22) conducted by the beginning of the workshop. The overall satisfaction regarding the use of the AcciMap as shown in Fig. 4 shows the density line skewing to the right indicating a considerably more than average probability of satisfaction. It should be noted that the results only reflected responses from thirteen participants from the initial fifteen that were involved in the exercise. The remaining two participants left during the focus group discussions and so their responses were not taken into account.

Fig. 4
figure 4

Histogram of the survey data from participants

Full size image

9.2 AcciMap results

The AcciMap outputs produced by each team A (Fig. 5), B (Fig. 6) and C (Fig. 7) based on initial observation showed similarities and differences in contributing factors identified. Each AcciMap output and safety recommendations were compared with one another and also compared with findings obtained from external (expert) review of the case incident. Contributing factors identified that were based on evidence are denoted as regular boxes while factors that are considered inferences are denoted as broken boxes. A qualitative reliability and validity assessment of outcomes produced by participants from their respective teams was carried out and will serve as part of the discussion section.

Fig. 5
figure 5

Team A—AcciMap output of case study

Full size image
Fig. 6
figure 6

Team B—AcciMap output of case study

Full size image
Fig. 7
figure 7

Team C—AcciMap output of case study

Full size image

Based on themes regarding contributing factors identified by each team, similar factors were commonly identified but expressed using different wordings. Their respective findings relating to contributing factors identified are discussed below:

9.2.1 Contributing factor: hospital’s computing systems

The contributing factor relating to the electrophysiology laboratory system was also identified by all and denoted as “incompatibility of IT systems” (Team A), “Electrophysiology system and hospital’s main computer not communicating with one another” (Team B) and “separate computer systems not communicating with one another” (Team C). Team B regarded issues relating to technology (Computer systems) not communicating with the one another (Hospital's main computer system) and how this contributed to the patient being misidentified (“Morrison” being confused with “Morris”). Team C also indicated issues relating to computing systems not communicating with each other leading to patient’s identity not being confirmed before the procedure was to commence.

9.2.2 Contributing factor: patient misidentification and communication issues

Contributing factors identified by Team C attributed to issues relating to “patient identification” and “patient being ignored by the physician”. These were as a result of communication issues relating to identifying the correct patient (staff not verifying the identity of the patient e.g. using the date of birth). One of the participants (Team C) pointed out this factor based on personal experience in reference to a patient (ward):

“Every time the ward was handed over, they read the ward’s date of birth”.

This would be considered a barrier against misidentification ensuring that the patient being examined is the right one. They further reasoned that failures exist when it comes to identifying names of patients even when two patients may be in completely different hospital areas and the barriers that should be in place to prevent it. Team B also identified the issue of human error relating to patient identification. Regarding the patient being ignored by the physician, participants from Team B reasoned that this was as a result of other factors including “pressures of waiting time” or “inadequate training”. There were no indications among Team A participants regarding communication issues but they indicated different staff (operations) failing to correctly identify the patient, similar to Teams B and C.

9.2.3 Contributing factor: patient’s uninformed consent

One of the contributing factors while differently worded carried the same meaning and identified by teams A and B was the patient giving an uninformed consent. Team A’s analysis, for example, depicted this contributing factor as “lack of informed consent” to which one of the participants noted an issue of consent relating to the patient:

“lacks the whole human factors elements to it; overburdened and exhausted physicians, they do not know the patients, they don’t know if they actually spoke about what the procedure is”

Another participant (Team A) supported this as a contributing factor and explained further that:

“Patients cannot frequently recall within hours of giving crucial information. But if we know that, why was it not getting spoken about earlier”

Their observations would explain their reasoning on the mix-up regarding the patient giving a consent where she consented but was not properly informed about the type of procedure she was going to undergo. The fact that the patient also experienced nauseating symptoms created a situation where assumptions were made about her condition leading to the belief that she was indeed supposed to undergo such procedure. The participants also appeared to acknowledge that the issue of consent regularly occurs in health practice although it was not explicitly stated in the case incident. Team B participants similarly identified “patient giving an uninformed consent” as elaborated by one of the participants:

“I assume that in most healthcare establishments, when a patient says no I do not want that (procedure), it happens, they pause and will not continue with the surgical operation”

The concept of “surgical pause” was considered a contributing factor but the question noted was if this was sufficient for the event to take place. Another participant even considered this as one of the holes in the Swiss Cheese! This was also shared by another participant who indicated the need for absolute clarity regarding the “pause” in the process and to look for certain indicators to get a green light as for whether to proceed with double-checking if it is the right patient for the procedure. This also includes the need for double-checking the consent form, getting the paperwork right, checking if the patient understands the procedure and evaluating whether it is safe to carry out the operation. If these indicators are not present, then the procedure should not even progress. In other words, the team determined that there were huge margins of failure in the system. However, Team C did not include this contributing factor explicitly but seemed to suggest from the factor “staff not listening to the patient and was not in agreement” that there was an issue with consent because the preceding cause leading to that effect was that there were “inadequate policies regarding patient consent” (organisational factor).

9.2.4 Contributing factor: organisational issues

While participants were allowed to make inferences on contributing factors at the organisational level, they also identified factors based on the information available in the case study. For example, Team A identified contributing factor “tolerance of lack of systemic identity checks” as a safety cultural issue relating to the hospital’s failure to conduct patient identity checks at different instances (at the physical level). Although, this was noted as an assumption and not necessarily a fact. Team B identified “Management complacency” as an organisational issue and their reasoning behind this factor was highlighted by one of the participants (Team B):

“Allowing the staff to take unilateral decisions when they shouldn’t as long as nothing goes wrong, then they are quite happy for that to let it go on”

Several other organisational contributing factors were identified by Team C including “Staff not challenging” was causally linked to another factor at the physical level (“study arranged despite no written order”). Only Team C had contributing factors including “lack of safety culture”, “lack of clinical governance” were identified as inferences rather than actual factors. They considered these factors to be reasons that contributed to the organisational culture of developing shortcuts and workarounds.

9.2.5 Contributing factors: external issues

No contributing factor based on evidence in the case study was identified by any of the teams for the external level but inferences were made. These inferences made by each team can be summarised based on their AcciMap outcomes in Table 12 below:

Table 12 Contributing factors (Inferences) based on the Case study (Teams A, B, & C)
Full size table

Reasonings behind each team’s decision to include these contributing factors were not openly discussed in their analyses but it does indicate differences based on their perception and understanding regarding possible systemic factors that created the climate for the series of events to occur.

It was observed during the exercise that some of the participants employed human factors thinking as well as traditional techniques, such as the 5-Whys and barrier analysis, based on their investigation experiences. Comparing the AcciMap outcomes by placing immediate causes after the incident at the “Physical Actors and Processes” level, we noticed similar events from the teams (particularly from Teams B and C). However, the causal linkages constructed by each team appeared to show differences. These will be discussed further in the following section of comparative analysis of results.

9.3 Comparison of AcciMap results

Each AcciMap result produced by respective teams was compared to assess the reliability (consistency) of outcomes produced. Based on the qualitative evaluation approach adopted by Branford (2007), the following criteria were used to evaluate the results:

  1. (1)

    Identification of causes/contributing factors at each level

  2. (2)

    Placement of causes/contributing factors at each level

  3. (3)

    Causal links (relationships) within and between each level

While it was not expected that each team would produce exactly the same output, the purpose was to determine if similar factors were identified, the level they were placed and if similar causal links between them can be identified. Also, wordings used to describe events (contributing factors) were not expected to be identical as long as they portrayed similar meanings.

9.3.1 Identification of causes/contributing factors

9.3.1.1 Physical actor events, process and conditions

At the physical/actor events level, Team A identified three (3) factors, Team B identified thirteen (13) and Team C identified seven (7) factors. The only common factor identified by all teams was the contributing factor relating to patient identification. Other contributing factors identified at this level between Teams B and C include the patient’s wishes being ignored by medical personnel and assumptions relating to a written order. Contributing factors distinctly identified by each team are summarised in Table 13.

Table 13 Contributing factors distinctly identified by each Team (A, B and C) (Physical Level)
Full size table
9.3.1.2 Organisational

At this level, Team A identified eight (8) causes/contributing factors, Team B identified seven (7) and Team C identified eight (8) although out of that number, four (4) indicated as “assumptions or inferences”. Similar contributing factors identified between different teams included issues relating to communication between computer systems (Teams B and C), staff not challenging authority and deficient handover processes (Teams A and C). Although the last factor was denoted as inference by Team C rather than as a valid contributing factor.

9.3.1.3 External

The case did not contain explicit evidence as to contributing factors, but the ones (inferences) identified by respective teams included issues relating to “legal implications regarding informed consent” (Team A), “waiting times and targets” (Team C) and “budgeting issues” (Team B). These were all identified as inferences.

9.3.2 Placement of causes/contributing factors

Identification and placement of causes/contributing factors at the appropriate level are very important in addressing areas of the system that needs to be improved through safety measures (Branford 2007). Contributing factors attributed to patient misidentification, miscommunication with the patient, patient giving uninformed consent and assumptions between various staff were all placed the appropriate level (physical actors’ level). Another factor; lack of communication between computer systems particularly between the main hospital and the electrophysiology unit was appropriately placed at the organisational level by Teams A and C. However, Team B had placed that particular contributing factor at the “Physical level” instead of at the organisational level as this will be considered to be within the control of the hospital organisation. Contributing factors identified by respective teams at the organisational and external levels while varied were appropriately placed.

9.3.3 Causal links within and between AcciMap levels

This criterion is perhaps the most challenging when comparing causal linkages between the causes/contributing factors identified from all teams AcciMap outcomes. Team B’s outcome had the most causal links (31), with team A having fifteen (15) and team C having twenty-one (21) links. Based on all causal relationships depicted by all teams, the only causal relationship that was similar was in respect to patient misidentification leading to the effect of the wrong patient being administered a procedure (Teams A, B, and C). Other causal relationships at the physical level that were similarly identified between different teams include:

  • The linkage (direct and indirect) between “Patient giving uninformed consent” and “Patient receiving a wrong procedure”—Teams A and B

  • The linkage (direct) between “Wishes of patient ignored” and “Patient being given an EP procedure”—Teams B and C

  • The linkage (direct and indirect) between “Staff not challenging/questioning authority” and “Patient being given a wrong procedure”—Teams A and C

At the organisational level, linkages identified by teams included more differences in terms of cause and effect. The only link that seemed to be similar between Teams B and C was between patient computer systems (electrophysiology lab and main hospital) not communicating and patient identification issues by different staff. Although for Team A, there was a direct link between computer system communication issues and the patient receiving a wrong procedure. There were no similar links identified between the external and organisational levels.

10 Discussion

10.1 Application of the AcciMap approach

Despite neutral responses to some of the questions in the survey, participants generally indicated an understanding and considered the AcciMap approach suitable for incident analysis. However, participants also shared and discussed recommendations from the retrospective analysis to identify similarities. Regarding the mapping of the causal relationships between each level of the AcciMap, one participant noted some difficulty in understanding the role of ‘actors’ at the external level in contributing to the accident. The participant questioned the benefit of analysing systemic factors at the external level especially in regard to if recommendations developed will improve system safety. Though this point was not supported by another participant in Team C who believed that by analysing the external level, one can determine possible latent conditions/weaknesses that enabled such an event to occur. Organisational culture and inadequate systems were generally considered issues from the incident particularly from team C. One of the participants noted that while this incident is only a “window”, believed that the next step an organisation needs to take is to determine if this is a systemic issue.

One of the participants (Team A) opined that it would have been preferable to implement the use of the AcciMap approach in their organization’s clinical incident scenarios. This point highlights the need for further investigation into the suitability of the AcciMap approach as a tool for clinical investigation in NHS boards. Two participants had previous knowledge on the use of a cause and effect template based on another type of systemic accident approach; ATSB (Australian Transport Safety Bureau) which was adopted in their NHS practice. Their experience in using this approach may have contributed to how they approached their analyses in their respective teams. Some participants utilised their experiences by applying RCA techniques like barrier analysis and 5-why(s)  in determining contributing factors. However, this was not considered a limitation in their respective analysis of the incident.

10.2 Method usage characteristics

Based on each team’s respective outcomes and survey results, their perception of the AcciMap approach is discussed based on the usage characteristics framework (Underwood and Waterson 2014).

10.2.1 Graphical representation of the accident

During discussions, participants generally agreed that the use of the AcciMap approach as a graphical tool can potentially help investigators to depict and identify specific problem areas that compromise patient safety. From the survey result, a high percentage of participants either “agreed” or “slightly agreed” that the graphical representation of the accident can serve as a useful means of communication (question 18). Only one participant slightly disagreed with this point. Another participant noted that the mapping of contributing factors provides a helpful way of promoting discussions with higher management. However, another participant indicated that AcciMap diagrams can become too complex unless contributing factors i.e. communication, staff competence are grouped under a higher hierarchy. In terms of representing the timeline of events as specified in the case study, participants (6) generally disagreed regarding diagrammatically denoting timelines of the events while the remaining participants were neutral in their response.

10.2.2 Data requirements

One of the participants commented on the nature of the case study and believed this incident was not something they experienced in their practice. The quality of the case study can also play a crucial role in how each of the teams interpreted the information of the events that led to the patient being given the wrong procedure. While the participants were guided using the table of contributing factors, they generally had varying views regarding systemic factors (organisational and external levels) that contributed to the adverse event. From the outcomes, there was an indication that they had challenges in determining systemic factors at those levels (organisational and especially at the external level).

10.2.3 Usability/ease of learning

The results from the survey data indicated that participants [slightly agreed (4), Agreed (2), and Strongly Agreed (1)] found the approach easy to use (question 16). The remaining participants (6) provided neutral responses. There were also neutral responses regarding how easy it was in understanding the AcciMap approach, and if it can be used for accident investigations in their NHS boards (questions 11 and 10). Participants collectively agreed that like any analytical tools, understanding and using the approach effectively depends on the skills, knowledge, and experiences gained from previous investigations. This also highly suggests that more time invested in training will be needed to effectively use the AcciMap approach. This last point was particularly emphasised by one of the participants during the workshop. During the exercise, the participants generally did not indicate difficulty in following guidelines regarding the placement of contributing factors in the appropriate AcciMap levels. However, it seemed the challenging aspect of the exercise was in the mapping of logical casual connections between each of the levels. Based on question 15 of the survey, their responses were mixed but from the subsequent discussion, some of them indicated the use of the AcciMap approach to be time-consuming even though they completed their analyses within the time frame for the exercise. 

10.2.4 Reliability of analysis

Based on observation of AcciMap outcomes produced, contributing factors identified at the physical actor process level appear to be more consistent between teams than factors identified at the organisational level. Contributing factor relating to scenarios where patient’s identity was not confirmed by different medical personnel was encapsulated as a single contributing factor “operation(s) failed to positively identify patient” by Team A. This was also applied to another factor identified by Team A; “operations making “assumptions” given the current information available”. Teams A, B and C identified factors relating to the misidentification of the patient but only factors relating to “patient’s wishes being ignored” and “staff did not listen to the patient and was not in agreement” were similarly identified by both Teams B and C. Other contributing factors that was identified by at least two teams related to “Computer systems (between Electrophysiology and Main Hospital) not communicating” (Teams B and C) and “Inadequate handover processes/procedures” (Teams A and B). Other organisational factors identified were related to staff members not challenging hierarchy (Teams A and C) and patient consent issues (Teams A and B).

10.2.5 Validity of analysis

Contributing factors identified from each team’s respective AcciMap output were compared to the AcciMap output independently produced based on an external review of the incident as shown in Fig. 8. Similar contributing factors at the physical and organisational levels were identified between all teams and external review as a form of face validity. However, a contributing factor that may be identified between the teams and experts' analysis may not be considered a valid factor especially since using expert review serves as the next best alternative in the absence of a gold standard measurement (Branford 2007). Participants slightly agreed that the AcciMap approach was effective in analysing contributing factors relating to technical components, human factors, organisational and environmental issues. However, concerning external factors, there seemed to be a contradiction between the survey result and their AcciMap outputs. This can be attributed to the lack of substantial information regarding external systemic factors. The number of neutral responses regarding sub-questions (6a to 6e) could be attributed to several factors including their first use of the approach in analysing this type of incident. In addition, being first-time users, the participants will need to review their analysis as they gain more understanding of the issues relating to why this kind of failure occurred. Regarding the AcciMap’s suitability for analysing accidents, there was a general agreement amongst the participants with only two of them being neutral. This observation corroborates with group discussions regarding developing causal connections when linking an adverse event(s) to the causes associated with it. It was further noted and agreed with the participants of the need for further training and understanding of the approach to improve their skills. This will be an important process to be able to improve content and face validity.

Fig. 8
figure 8

AcciMap output of case incident—expert review

Full size image

10.3 Focus group discussions

The concluding part of the workshop was spent discussing their opinion on the benefits of applying the AcciMap approach based on their first-time experience. Participants were generally in agreement that the AcciMap approach was relatively easy to understand and use for incident analysis. Although, several participants expressed the need for more time to fully understand the case study and more training to fully implement the method. In terms of the advantages of the AcciMap method, one participant remarked:

“You can see relationships between various factors at various levels”

One participant (Team C) opined that in addition to the contributing factors table listed in the AcciMap training manual, other potential contributing factors identified need to be grouped similar to how the fishbone technique is used to identify and classify factors into different categories. Another participant (Team B) also supported this point and further believed that the process will require brainstorming regarding contributory factors and mapping them using the AcciMap approach. However, one of the demerits noted by another participant was the issue of the approach not having timelines and in particular the non-inclusion of barrier analysis. While it was explained to them that the case study had a timeline of events that happened, a participant (Team A) further explained:

“I suppose we struggled a little bit because that’s getting you to your contributing factors but not to your barriers”.

This point particularly focused on whether the consent policy existed or whether it was not followed and that if it was the former, then it would have been considered as a missing barrier. Participants generally agreed regarding the subjectivity of the outcomes and the need to improve its reliability after they had reviewed their outcomes. There was also a general consensus regarding the AcciMap approach being utilised as a potential toolkit for accident investigations in NHS practices. However, several participants indicated that the approach can only be considered part of a toolbox in conjunction with existing traditional RCA techniques, such as timelines and the fishbone technique.

10.4 Comparing safety recommendations

Based on safety recommendations produced by each team, there were few similarities and varying measures based on respective analyses (see Table 14). This is particularly evident where each team developed its safety recommendations based on step 9 from Branford’s AcciMap guidelines. Similarities from recommendations produced include implementation of safety briefs (Teams A and C), reviewing of processes relating to consent policy (Teams B and C) and reviewing of existing systems (Computer) (Team A and B). These recommendations relate to contributing factors including handover processes, communication relating to computer systems and patient consent policies.

Table 14 Safety recommendations of Teams A, B, and C based on Wrong Patient case study
Full size table

Comparing each team’s safety recommendations with those produced from external analysis (see Table 15) shows similar outcomes relating to patient consent policies and reviewing computing systems in terms of synchronising with each other with updated patient information. However, other recommendations not indicated by respective teams include organisational safety culture in regard to challenging hierarchy when reporting concerns and reviewing of policies and training regarding patient identification.

Table 15 Safety recommendations (external analysis) on the wrong patient case incident
Full size table

11 Limitations of the study

Conducting training and evaluation of the formalised AcciMap approach with NHS participants had its challenges. The length of time assigned for the workshop particularly for the analysis exercise was not ideally sufficient for the analysis of the case incident. Despite the case information being given provided for the participants before the workshop, they still needed to refer to aspects of the report and this affected the time allocated to not only complete their analyses but also review where necessary. While this study did not focus on the amount of time it took for teams to complete their analyses, it is worth noting that this could have an impact on the consistency and validity of their respective outcomes. This was very much reflected in the survey result regarding completion of analysis within designated time (question 20) and if the approach is time-consuming (question 21). If more time was allocated, it would have allowed each team to review their initial analyses, identify any missing information regarding contributing factors not included in the diagram and refine their outcomes. This can be achieved as a formal process called intra-reliability where the AcciMap approach is applied again on the same incident by the same set of participants to compare if they produce consistent outcomes (Goode et al. 2017).

Another limitation involves the nature of the case incident as has already been highlighted in this study. Producing accident case reports that carry comprehensive information on contributing factors not only about actions of individuals but also organisational and external factors are needed to identify system weaknesses. Lack of such information from this incident report used potentially contributed to the challenge of participants identifying contributing factors. One of the challenges of applying this type accident approach to case reports that do not contain detailed information from all socio-technical aspects of the healthcare system is that it can be difficult to identify systemic weaknesses that contributed to human errors. This case incident was initially carried out using RCA techniques which do not allow for a deeper analysis of other factors that played a role. While this incident took place in the USA, it was also interesting to note from a few participants regarding how new this type of incident was to them as they have never experienced such type in their practices. This could have possibly contributed to how participants analysed the incident due to unfamiliarity and how things work in UK health settings compared to their US counterparts.

AcciMap results produced from the teams also indicated that despite the number of participants per team, there were contrasting contributing factors identified at each AcciMap level. These differences can occur as a result of their understanding of the case study, contributing factors each team could agree on to be considered valid and determining appropriate levels and causal relationships. It was also challenging to conduct an immediate follow-up of the workshop with participants to further elaborate their reasonings behind their initial findings due to their commitments in respective boards and availability.

12 Conclusion

Based on survey data, comparative analyses of AcciMap results and discussions with participants, there was a general appreciation of the benefits of adopting the AcciMap approach for incident investigations and analysis. Aspects of the usability of the approach including its ease of use, serving as a communication tool, and fostering team collaboration were positive for the participants. However, aspects relating to its ability to identify contributing factors relating to socio-technical aspects received generally mixed responses. The reason for this can be attributed to their understanding of the incident and the time restriction which may have prevented them from thoroughly reviewing their respective outcomes. Regarding the potential for the AcciMap approach to be adopted for incident analysis in clinical practices, a crucial aspect noted in the workshop was the AcciMap approach being time-consuming. This factor may have influenced how they regarded suitability of the AcciMap approach for accident investigations compared to their experience using RCA techniques. The ability to use the AcciMap approach for clinical investigations requires not only knowledge of the domain but also requires users to correctly apply the guidelines and where necessary, update the initial analysis to produce the final result. However, these processes can a considerable amount of time and effort, especially in a domain as complex as healthcare considering limited resources.

Aspects relating to reliability, and validity are also very important to practitioners for the approach to be considered a valuable toolkit. As was indicated in the focus group discussion sub-section, a suggestion made by one of the participants regarding the grouping of contributing factors into different hierarchical headings suggests a need for developing a more structured approach of classifying contributing factors across different levels. As suggested in Waterson et al. (2017) regarding the evolution of the AcciMap approach, reliability and validity can be improved potentially by combining with other existing techniques. Based on contributing factors (inferences) identified by participants at the organisational and external levels, themes can be extracted from case reports that can allow an AcciMap approach specific to the health domain to be developed. Other aspects noted in their experience of using the AcciMap approach included the lack of incorporation of timelines and being able to use barrier analysis for determining existing barriers relating to contributing factors.

It is important to acknowledge that as of the time this workshop was conducted, the concept of systems thinking and approaches incorporating it was still relatively new. Since then, a growing number of NHS practices are gradually adopting systemic approaches for incident analysis. The AcciMap approach itself needs to be further explored as to how it can be further improved to be considered a valuable tool. The work of Salmon et al. (2017) achieved this through the development of a classification scheme based on the underlying AcciMap framework but specific to led outdoor activities. This approach is currently being applied in the development of a proposed health-specific AcciMap approach and will be evaluated with safety practitioners in NHS for its reliability and validity.