‘It’s a State of Mind’: a qualitative study after two years’ experience with the World Health Organization’s surgical safety checklist

Based on current weaknesses in the understanding of the mechanisms, factors and positive effects involved in checklist usage, as well as a scarcity of qualitative approaches, the aim of this study was to explore surgical personnel experiences with the World Health Organization’s Surgical Safety Checklist, 2 years after implementation. Three focus group interviews were conducted with surgical personnel. An interview guide was designed to assist the interview process, comprised of broad, open-ended questions. Notes and audio recordings were fully transcribed verbatim and subsequently analysed using qualitative content analysis in order to identify emergent meaning units, categories and themes. The checklist improves confidence, team communication and sharing of critical information in the surgical team. However, informants described the occurrence of wrong-site surgery not prevented by the checklist due to preoperative wrong-site marking combined with automated checklist use in the operating room. Using the checklist as a ‘tic box exercise’ was recognized as a safety challenge by all professions, especially during routine surgery. The surgical team balances safety and effectiveness constantly in the operating room. Challenges that need to be addressed include making the Sign-in part a team effort and taking the accompanying pause in performance during the Time-out and the Sign-out, with cross-check of items, in order to avoid automated checklist use. We suggest surgical team training that includes checklist performance, in order to enhance the quality of checklist use.


Introduction
Recent health research literature on surgical checklists reports that the introduction of surgical safety checklists has reduced mortality and morbidity in surgical populations (Haynes et al. 2009;Weiser et al. 2010a;de Vries et al. 2010). The World Health Organization (WHO) introduced the Surgical Safety Checklist in 2008, designed and developed to improve surgery globally (Weiser et al. 2010b). The checklist consists of a Sign-in, a Time-out and a Sign-out part, performed at three critical steps in the surgical pathway. The WHO has defined ten aims for the checklist: avoiding wrong surgery, safe anaesthesia (drugs, monitoring, airway, allergies), being prepared for blood loss, minimizing infection risk, avoid retaining of instruments, correct sample labelling, safe communication and monitoring of surgical volume and quality (WHO 2009).
Wrong-site surgery is preventable (DeVine et al. 2010). However, in a systematic review of the literature, DeVine et al. did not find evidence in support of Timeout protocols such as the Joint Commission Universal Protocol leading to a decrease in the rate of wrong-site or wrong-level surgery (JCAHO 2001). This finding is in accordance with studies by Lee (2010) and James et al. (2012). However, Lee (2010) also found that the extended Surgical Time-out before anaesthesia induction improved team communication and timeliness of antibiotic administration and did not disrupt work flow. In another study by Haynes et al. (2009), the authors conclude that even though the WHO's Surgical Safety Checklist can reduce mortality and morbidity, the mechanisms behind such positive checklist effects were not discovered. In a recent study of changes in safety attitudes after introduction of the WHO's Surgical Safety Checklist, Haynes et al. (2011) suggest that there is a correlation between positive changes in attitudes and perceptions of teamwork with improved patient outcomes after implementation of the checklist. Furthermore, Haynes et al. (2011) suggest that studies aimed at exploring the changes and effects of checklist usage, in order to reveal the factors involved, require both qualitative and quantitative approaches. A qualitative approach was employed in an ethnographic study of surgical teams in a Norwegian tertiary hospital, aimed at exploring the experiences with the Surgical Safety Checklist immediately after implementation (Hollund 2010;Høyland et al. 2013). The study revealed that perceptions and attitudes regarding the application of the Surgical Safety Checklist varied between the professions. The surgeons were most reluctant to use it, underlining a need for change in culture to reach the safety improvement potential of the checklist in practical use. The study also revealed that not performing the Sign-in part of the checklist as a team, i.e. deviating from guidelines, deprived them of an important safety barrier and made the operations more vulnerable to wrong surgery. On the other hand, the checklist improved team communication and the nurses' confidence levels in the operating room. The checklist also triggered a feeling of being an integrated part of the team.
The studies described above suggest that there is a need for qualitative studies that can provide a deeper insight into the nature of checklists. Thus, the aim of this paper is to explore surgical personnel's experiences with the WHO's Surgical Safety Checklist, 2 years after implementation in a surgical unit in a Norwegian tertiary hospital.

Design
A descriptive qualitative study design, comprising focus group interviews and qualitative content analysis, was carried out in a Norwegian tertiary hospital. The focus group methodology is used to collect qualitative data, specifically by engaging a small number of informants in group discussions around a particular topic or set of issues (Krueger and Casey 2000). The strength of this methodology lies in the informal nature of the data collection, as the moderator encourages interaction between group members instead of asking questions to each participant. This creates an atmosphere of openness for participants to explore issues they perceive as important (Krueger and Casey 2000;Hammersley and Atkinson 2007). Hence, the group methodology facilitates an interactive discussion between research participants and reflection unique for the group context (Kitzinger 1994). The multidisciplinary nature of the Surgical Safety Checklist necessitates all relevant professions views on the subject, resulting in three interview sessions, as described below.

Sample
The study was carried out over 3 months in 2011 and 2012 at a tertiary hospital serving a population of 600,000 as a referral hospital. All types of surgery are performed except for transplant surgery. The recruited informants were sampled to elicit the particular surgical or anaesthesia professional views, resulting in three homogenous focus groups of nurses (operating room nurses and nurse anaesthetists), consultant anaesthetists and surgeons. The informants were selected through consultation with the unit managers and the researchers (ASH, ØT). There is no consensus of how many participants a focus group interview should contain; Krueger and Casey (2000) suggest that a smaller number of participants show greater potential. Our study sampled three operating room nurses and three nurse anaesthetists (group #1), four anaesthetists (group #2) and four surgeons (group #3). To ensure representativeness, the informants were selected to cover variety across age groups, gender and experience as senior consultants (Table 1). The main inclusion criterion for participants was usage of the WHO's Surgical Safety Checklist over more than 1 year.

Interviews
An interview guide was developed for the focus group interviews (Supplementary file 1) and structured according to elements in Kyrkjebø et al. (2006). The interview guide focused on two themes: the experiences with the Surgical Safety Checklist and the nature of operating room (OR) time. Each interview session began with a presentation of participants and the researchers' roles, and handouts of study information, including clarification of data preservation/anonymity issues. The opening and main question ('core trigger' 1 ) during each theme was broad and open ended, facilitative of group interactions. As the moderators deemed it appropriate, additional triggers were introduced to stimulate the discussions. At the end of each interview session, the moderators summarized their impressions and encouraged comments/feedback from the informants. For all three focus group sessions, ASH and SH were moderators on the checklist and OR on the time theme, respectively, while ØT was co-moderator during the first interview and pilot sessions. Specifically, a pilot interview containing two anaesthetists was conducted evaluating the methodology and performance of the moderators. The focus of this paper is the data gathered on the checklist theme, concerning experiences with the Surgical Safety Checklist.

Data collection and analysis
The interview sessions were audiotaped and subsequently verbatim transcribed by SH. A systematic qualitative content analysis was then performed on the focus group interview data pertaining to the checklist by ASH and SH. Specifically, the transcribed text was coded into meaning units, followed by related categories and themes, in line with Graneheim and Lundman (2004). Sentences and phrases related to the aim-to explore surgical personnel experiences with the WHO's Surgical Safety Checklistwere understood as meaning units. These meaning units were further condensed to subcategories and categories to discover patterns in the data. Two of the researchers (ASH and SH) went independently through the coding processes. The final step of the content analysis involved comparing the coding for relationships and accordance across themes and categories. This produced variations and slightly different coding results. Specifically, the researchers identified numerous text segments (ASH = 96, SH = 108), which were processed into meaning units, and then categories. Concerning the focus on commonalities, researchers conducting independently the analysis of the data should preferable achieve a common understanding of more than 80 % agreement on the coding (Bradley et al. 2007). Accordingly, our coding process revealed a satisfying agreement with a variation of 11 % (96/108) in the text segments identified. The most notable variation in coding was related to ASH's degree of contextual surgical insight, complementing the safety insight of SH.

Ethics
Based on approval from the Norwegian Social Science Data Services, all participants were handed a written information form on study aim, data preservation, voluntary consent, anonymity issues, etc., and this was also clarified verbally by the researchers during the introduction phase. All participants agreed to the study, by signing a field for consent provided in the form. Table 2 provides an overview of the meaning units, categories and themes resulting from the content analysis described above. The condensed themes 'the checklist is optimizing safety' and 'safety and effectiveness are balancing in the OR' supported by a number of categories and meaning units, emerged as the main findings of the study. These categories, with representative quotations, are described next. 3.1 Optimizing safety 'The checklist optimizes safety' theme was derived from four condensed categories, reflecting the upsides of the checklist: 'quality in performance of the checklist enhances safety', 'the checklist is improving safety culture', 'the checklist improves communication, openness and confidence' as well as 'the checklist improves information handover'. Excerpts supportive of the category 'quality in performance of the checklist enhances safety' includes:

Results
I believe the Time-out has influenced the team, in terms of better contact within the team. We all start a conversation before surgery; it provides the opportunity to ask questions about equipment and the procedure. I think it is an improvement (operating room nurse) My colleagues recognize the checklist as a necessity, we need it (surgeon) It feels unsafe not using the checklist once you are familiar with it (anaesthetist) The category 'the checklist improves the safety culture' can be seen in quotations such as: The checklist is a state of mind, not a piece of paper (anaesthetist) The checklist is an extremely good thing; we have really become sharpened after its introduction. I believe it is very positive (nurse anaesthetist) It is very useful to check patient identity, correct site, x-rays, since the fear is one day to perform an unforgivable error. In this regard, the Time-out makes you remember or trigger thoughts on safety issues (surgeon) Checklist saves OR a time 9 7/taking time to a proper time-out improves safety 9 4/ challenging surgery or experienced OR nurse lead-enhances checklist performance/thorough performance from OR a nurse is vital 9 2 Safety culture Checklist might have generated improved general safety focus in the organization/the checklist constitutes positive change in culture/checklist sets the agenda for the operation/surgeons are more loyal now/Wrong-site surgery-wake up call for checklist quality performance/recognize that automated checking is unsafe/ improved checklist performance is demanded Communication, openness and confidence The checklist improves team communication, openness and confidence 9 6/checklist is positive for us/time-out presentation is positive for new personnel/personal engagement-eager to perform well Information handover Sign-out part is of special importance for information handover to postoperative care/ checklist reduces postoperative questions about prophylactic antibiotics and thrombolysis 9 3 Balancing safety and effectiveness Challenge to pause during time-out/sign-out Checklist is performed without attention 9 11/rushes through checklist to save time 9 9 Automation of checklist performance Automated checklist performance being related to examples of wrong-site surgery Effectiveness priority over checklist use Not performing checklist to save time 9 3/surgeon focuses on tasks and not checklist 9 5/surgeon performs the checklist by memory 9 2/the presentation of team members is less frequently performed during time-outs 9 2/effectiveness and safety balances/sign-out commonly performed as the surgeon is suturing Changes in safety routines After the checklist implementation, new procedures and surgical stop orders are introduced for site marking/signing/nametags during sign-in/personal engagement Emerging safety debates in the surgical teams Site marking and checklist performance/to stop working and pause during the timeout/who should be leading the sign-out check/achieve consensus about the importance of safety check items Debating checklist overlap with pre-anaesthetic checklist or implants available at time-out, whom to perform the sign-in and sign-out/not being an essential part of the checklist team and absence during time-out and sign-out (anaesthetist)/call for training in communication, checklist use and team management in OR a Another category found to optimize safety was that the checklist seems to enhance 'communication, openness and confidence' in the OR, as seen in the following quotations: The Sign-in has become more thorough and extended, since we are now signing for nametags and site marking. Particularly the identity check has improved after we recently had a case of wrong patient in the OR, which was discovered by a coincidence (nurse anaesthetist) I have to say that sterile assisting has definitely become more interesting after introduction of the checklist. I feel more as a part of the team (operating room nurse) Both surgeons and anaesthetists are dependent on active and determined nurses to lead the checklist performance (anaesthetist) 'Information handover', both in the OR and to the post anaesthetic care unit (PACU), represent the final category, supported by quotations such as: 'The checklist has generated changes in the PACU. It contributes to a significant reduction in questions about prophylactic antibiotics and thrombolysis' (anaesthetist) I believe that the checklist functions well, but I am especially fond of the summary in the Sign-out (anaesthetist) During each of the focus group sessions, we asked the informants if they would have wanted the surgical team to perform the checklist, should their next of kin or they themselves undergo surgery. All participants answered positively, further underscoring the safety optimizing function of the checklist in the OR.

Balancing safety and effectiveness
The second theme-'balancing safety and effectiveness'is supported by the categories: 'challenge to pause during Time-out', 'automation of checklist performance', 'effectiveness prior to checklist use', 'changes in safety routines' and 'emerging safety debates in the surgical teams'. Next, we highlight quotations supportive of these categories.
The ability to pause during the Time-out and Sign-out represented a challenge to surgical operations, as seen in these quotations: In routine surgery, the Time-out can be a litany; we do not pay enough attention (operating room nurse) Often people keep working on whatever they are doing; they do not pause and focus, and continue talking and working. Yet, it is more natural to do the checklist now (surgeon) In my opinion the checklist supports safety, but we need to become more attentive in performing it, and ask 'which side is it?' instead of 'is this the correct foot (operating room nurse) Sometimes, as the nurse anaesthetist is occupied with finding stuff in the trays and the surgeon has started to operate, the coordinating operating room nurse is unable to gain attention for the Time-out (anaesthetist) Effectiveness priority over checklist use was an issue among the informants across all professions, not necessarily connected to the checklist primarily but still described with checklist examples: Even if we do not hand over the knife or blade until after the Time-out, the surgeon starts drawing and are very focused on the next steps of the surgical procedure (operating room nurse) Time is always a factor, is the message that runs through our heads every day. One tends to perform as well as possible, but sometimes it just goes to fast (nurse anaesthetist) I agree that we often work very efficiently and fast and at the end of the day one feels that today we could have made errors as we cut corners on the checklist use (operating room nurse) All surgical personnel groups described that they to some degree adjusted the checklist performance to the flow of care in the operating room, automated checklist use, establishing new safety routines or safety debates among the surgical personnel. Some of these issues are described in the following excerpts: The list becomes more a task itself than a real checkpoint, something you have to do to get started on the actual job (related to automated checklist use and mix up of drugs) (anaesthetist) In general, I believe that the greatest danger with the checklist is if we perform it without attention (automated) (surgeon) Last week a patient had been site-marked preoperatively and we performed the checklist as usual. When we opened up his foot, we discovered it was the wrong one. So the checklist sometimes goes like (illustrated with sound-tic box) performed without thinking it through (nurse anaesthetist) In the beginning, they (coordinating OR nurse) went through the checklist as one should, point by point. Now you come in and just state it by memory, it is not good enough, really (surgeon) The findings suggest that the informants adjusted themselves to the fact that the checklist had become a standard of care. Adjusting to OR effectiveness, the nurse anaesthetist performed the Sign-in part on his/her own or together with a nurse anaesthetist, not together with the anaesthetist and the operating room nurse. Informants across all professions pointed out safety breaches of infrequent errors such as wrong-site surgery, wrong patients in the operating room, wrong drugs being administered and wrong journal papers leading to blood transfusion errors. The wrong-site surgery was explained as a result of preoperative wrong-site marking of the site combined with automated checklist performance. After 2 years' experience with the WHO's Surgical Safety Checklist, its safety barrier potential was recognized by all the informants. Thus, the safety checks are balanced with the effectiveness in the OR.

Optimizing safety
Our first main theme concerning the perception 'the checklist optimizes safety' corresponds to results in a Swedish survey conducted after 1 year with surgical Timeouts, where 93 % of the surgical personnel perceived that the Time-out increased patient safety (Nilsson et al. 2010). Specifically, our study suggests a positive change in that the surgeons, after 2 years use, recognize the importance of the checklist. This stands in contrast to findings in a previous study of the checklist at the current hospital immediately after implementation (Hollund 2010;Høyland et al. 2013), where the surgeons appeared reluctant to use the checklist. The expression 'the checklist is a state of mind, not a piece of paper' is particularly supportive of the safety potential of the checklist. The expression recognizes safety as a way of thinking and acting in relation to the checklist, within the medical domain. The expression also suggests that optimizing safety depends on the degree of internalization of the checklist, in the sense that the checklists need to be accepted by the organization, culture and end-user level before eventually, if willingness and maturity develop, 'a state of mind' can be reached. To improve safety, health care needs to get the technical and adaptive work right. Without attention to adaptive work, checklists would probably suffer the same fate as guidelines-often left unused, even when very robust (Bosk et al. 2009).
Further underscoring the role of the checklist in optimizing safety, our findings suggest that a thorough performance of the checklist enhances safety and improves communication, openness and confidence among the surgical personnel. These findings are reflected in the health research literature (Thomassen et al. 2010;Lingard et al. 2008;Lee 2010). Specifically, in a qualitative study of a pre-anaesthetic induction checklist, by Thomassen et al. (2010), the checklist was found to induce standardization and improve confidence in health care workers. In another study by Lingard et al. (2008), inter-professional checklist briefings are found to minimize communication errors and promote active and collaborative team communication. Lee (2010) also identified improved communication after introducing an extended surgical Time-out.

Balancing safety and effectiveness
The second theme identified in our study, 'balancing safety and effectiveness, while not inherently connected to the checklist use, was often exemplified by checklist practices. The essence of this theme is that there seems to be an ambiguity in the perceptions of effectiveness and safety among surgical personnel. On the one hand, this is expressed through efforts in achieving the goals of the surgical operating schedule in ways that show concern for patient safety, while on the other hand, 'shortcutting' the checklist performance is considered necessary to follow the program. The latter 'trait' makes the operation vulnerable to safety violations. In particular, the findings suggest that the Sign-in part of the checklist is still performed by one person alone or together with another nurse anaesthetist, thus not following the team intention in the WHO guidelines (2009). This safety concern was further underscored by Hollund (2010) and Høyland et al. (2013) as a possible challenge 3 months after introduction of the Surgical Safety Checklist. Lingard et al. (2005) suggest that if some team members decide to do the checklist on their own, other members might feel excluded. In other words, poor checklist use can influence inter-professional dynamics (Mahajan 2011). Our findings suggest that an individual's feeling of not being an essential part of the checklist performance could influence distribution of work tasks as well as actual participation during the checking.
Furthermore, automated performance of the checklist represents a challenge. To prevent this phenomenon in other domains, such as nuclear power plants, offshore petroleum production, civil aviation and military services, mandatory full-scale checklist training has been considered the gold standard for decades (Thomassen et al. 2011). The implicit suggestion is that team training in health care could improve checklist implementations. Supportive of this thinking, Vats et al. (2010) highlight three steps of importance to sustain quality use of the National Health Services Surgical Safety Checklist: developing local champions, supportive organizational leadership and surgical team training.
Summarizing, our findings suggest that although the focus on maintaining effectiveness of the surgical team encourages shortcuts in the checklist performance, there exists a compensating focus on maintaining the necessary safety efforts by ensuring a thorough checklist performance. The surgical team constantly balances on this safety and effectiveness line. We believe this balancing act is to a certain degree being challenged by automation of behaviour in the checklist performance and that this represents a concern.
4.3 Wrong-site surgery In our study, surgical personnel report the wrong-site surgery even with the use of the Surgical Safety Checklist, suggesting that the potential safety concerns observed in Hollund's (2010) and Høyland et al's (2013) studies in the same OR-setting immediately after checklist implementation still occur. The literature is supportive of this, in that surgical checklists do not appear to eliminate wrong-site surgery (DeVine et al. 2010;Lee 2010;James et al. 2012). Thus, our findings indicate that the WHO's aim to avoid wrong surgery through the Surgical Safety Checklist is not met in practice. Our findings suggest that when performing the checklist, the surgical team members doing so may not capture the attention of the team as a whole. The ability to stop and focus during the Time-out combined with the use of active communication could enhance the checklist dynamics. In particular, the safety items of patient identity, procedure and site must be cross-checked against patient information and records, planned procedure and X-rays if applicable, facilitated by active communication to prevent wrong surgery (patient, site or procedure). Literature on checklist use in aviation supports the idea of cross-checking the checklist items as a team effort (Degani and Wiener 1993), where the task of verification should not be left solely to the person responding to the checklist. The ability to cross-check during the Time-out could also be affected by how the surgeons and the team often rush through the checklist items to get started with surgery, as suggested in our study.
Memory-guided checklist performance and 'shortcutting' the checklist also represent challenges in the routine of calls and responses in aviation (Degani and Wiener 1993). This was also reported by all professions in our study, especially during the Sign-in check and the litany of the Time-out items without attention from the whole team. In medicine, prospective memory failures are recognized as related to patient safety with potential to harm patients (Dieckmann et al. 2006). This indicates a need for a systematic checklist performance-step by step-through every item on the list, as previously described by Hollund (2010) and Høyland et al. (2013). Mahajan (2011) calls upon team awareness and clinical engagement during checklist performance for realization of its safety benefits.
In sum, our finding that wrong surgery still exists despite the use of the checklist, possibly caused by poor Sign-in and Time-out phases which again could be related to the surgeons' and the team members hurry to begin surgery, demonstrates the ambiguity in terms of effectiveness and safety in the operating room.

Strengths and limitations
The informants were selected based on their experience with the checklist, and our interest in sampling the views of personnel across relevant professional groups on the application of the checklist. We believe the resulting variety in perspectives represents a main strength of this study. The perspectives of safety researchers and health researchers are supportive of other studies conducted within the surgical clinical setting (Høyland 2012). In terms of limitations, it is possible that including an interprofessional focus group mixing surgeons, anaesthetists and nurses would produce different results. Further, inclusion of a head consultant anaesthetist might have influenced on the willingness to speak up in the anaesthetist focus group. However, the results of the content analysis suggest that such effects did not occur.

Conclusion
After 2 years with the WHO's Surgical Safety Checklist, surgical personnel appears to have adapted to the checklist as a standard of care. All professions recognize the checklist as an element that enhances safety in the OR, and increasingly so as it has become an integral part of the daily routines. However, challenges that need to be addressed include making the Sign-in part a team effort, making room for pause in performance during the Time-out and Sign-out and cross-checking in order to avoid automated use of the checklist. In other domains, this has been met by regular and mandatory checklist training in full-scale simulators. Similarly, we suggest an emphasis on surgical team training, including checklist performance, in order to enhance the quality of checklist use.
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.