Surgical safety checklist: implementation in an ambulatory surgical facility
In 2007, the World Health Organization created a Surgical Safety Checklist (SSC) that encompassed a simple set of surgical safety standards. The threefold purpose of this study was to add ambulatory-specific items to the SSC, to introduce the items into an ambulatory surgical facility, and to determine if patient outcomes regarding postoperative pain and nausea/vomiting improved following implementation. In addition, safety attitudes, antibiotic timing, regional anesthesia/nerve blocks, preemptive pain medications, prophylactic antiemetics, length of stay, and hospital admission were also assessed.
After Research Ethics Board approval, staff complete a Safety Attitudes Questionnaire. Seven items were added to the SSC. Data were then collected on 180 surgical cases before SSC implementation and 195 cases following implementation. Compliance with each section of the SSC was assessed.
On postoperative day one, the median (97.5% confidence interval [CI]) difference between pre- and post-implementation pain scores was 0.5 (97.5% CI, 0 to 1; P = 0.13), and the median difference in the rate of post-discharge nausea/vomiting was −8.4% (97.5% CI, −17.9 to 1.1; P = 0.06). There was no improvement in safety attitudes or any of the secondary outcomes, with the exception of the use of preemptive pain medications. Compliance with the three sections of the checklist, i.e., BRIEFING, TIME OUT, and DEBRIEFING was 99.49%, 97.95%, and 96.92%, respectively. There was low compliance in verbalization of the added “ambulatory-specific items”.
Potential reasons for lack of uptake and integration include poor “user” buy-in, an overly lengthy checklist, and lack of prioritization of ambulatory-specific items. A shortened SSC was developed based on the results of this study. This trial was registered at ClinicalTrials.gov ID: NCT00934310.
La liste de contrôle de la sécurité chirurgicale et sa mise en œuvre dans un centre de chirurgie ambulatoire
En 2007, l’Organisation mondiale de la Santé a créé une Liste de contrôle de la sécurité chirurgicale (LCSC) couvrant un ensemble simple de normes de sécurité pour la chirurgie. L’objectif triple de cette étude était d’ajouter des éléments spécifiques au contexte ambulatoire à la LCSC, d’introduire ces éléments dans un centre de chirurgie ambulatoire, et de déterminer si les devenirs des patients en matière de douleur et de nausées et vomissements postopératoires étaient améliorés suite à la mise en œuvre de la LCSC. En outre, les attitudes face à la sécurité, l’administration opportune des antibiotiques, l’anesthésie régionale / blocs nerveux, les médicaments de prévention de la douleur, les antiémétiques en prophylaxie, la durée du séjour et les admissions à l’hôpital ont également été évalués.
Après avoir reçu le consentement du Comité d’éthique de la recherche, le personnel de l’établissement a rempli un Questionnaire sur les attitudes face à la sécurité, à la suite duquel sept éléments ont été ajoutés à la LCSC. Les données ont ensuite été colligées sur la base de 180 cas chirurgicaux avant la mise en œuvre de la LCSC et de 195 cas après sa mise en œuvre. Le respect de chaque section de la LCSC a été évalué.
Au premier jour postopératoire, la différence médiane (intervalle de confiance [IC] 97,5 %) entre les scores de douleur avant et après mise en œuvre était de 0,5 (IC 97,5 %, 0 à 1; P = 0,13), et la différence médiane dans l’incidence de nausées et vomissements après le congé était de −8,4 % (IC 97,5 %, −17,9 à 1,1; P = 0,06). Il n’y a eu aucune amélioration dans les attitudes face à la sécurité ou dans toute autre mesure secondaire, à l’exception de l’utilisation de médicaments de prévention de la douleur. Le respect des trois sections de la liste de contrôle, soit Réunion préparatoire, Temps mort et Réunion-bilan (BRIEFING, TIME OUT, DEBRIEFING) était de 99,49 %, 97,95 %, et 96,92 %, respectivement. Le respect était peu maintenu au niveau de la verbalisation des « éléments spécifiques au contexte ambulatoire » ajoutés.
Parmi les raisons potentielles ayant freiné l’adoption et l’intégration de cette liste améliorée, citons le manque de conviction des « utilisateurs », la longueur de la liste de contrôle, et le manque de priorisation des éléments spécifiques au contexte ambulatoire. Une LCSC raccourcie a été mise au point sur la base des résultats de cette étude. Cette étude est enregistrée sous ClinicalTrials.gov, ID: NCT00934310.
It has been more than ten years since the Institute of Medicine published the pivotal report, To Err is Human: Building a Safer Health System.1 The patient safety movement has made significant strides, and the development of the World Health Organization’s (WHO) Surgical Safety Checklist (SSC) was the first major international collaboration to improve perioperative patient safety. The SSC was piloted in eight hospitals around the world from October 2007-September 2008.2 The death rate was 1.5% before the checklist was introduced, and the rate declined to 0.8% afterward (P = 0.003). Furthermore, inpatient complications occurred in 11.0% of patients at baseline and in 7.0% of patients after the checklist was introduced (P < 0.001).2 Subsequent studies have supported the findings in this hallmark publication.3-7
The significant improvement in morbidity and mortality was shown in inpatient hospitals. During the past decade, however, there has been a 300% increase in the number of surgical/diagnostic procedures performed in ambulatory surgical centres in the United States. In ambulatory surgical centres, there is a much lower incidence in morbidity and mortality and a different set of complications.8 In a report by Kurrek and Twersky, the authors studied the literature published during 1997-2008 for incidences of mortality and morbidity in out-of-hospital facilities.9 Their findings indicated a mortality rate of 0.00008-0.05% and a complication rate of 0.7-1.3%.
Post-discharge nausea and vomiting (PDNV) is an important contributor to postoperative morbidity in the ambulatory surgical population.10 Similarly, postoperative pain is undermanaged, with as many as 86% of patients experiencing moderate to severe pain.11 The purpose of this study was to examine the effectiveness of the SSC in the context of an ambulatory surgical facility by adding ambulatory care-specific items to the checklist, assessing adherence to the safety checklist items, and examining the impact of the SSC on safety attitudes and important patient outcomes.
After Research Ethics Board approval, all operating room and postoperative anesthetic care unit personnel in an ambulatory surgical facility were asked to complete a 58-item validated Safety Attitudes Questionnaire (SAQ).12,13 Questions pertaining to communication and collaboration were assessed using a five-point scale with 1 = very low and 5 = very high, and the individual’s “experience” in the work environment was assessed using a five-point scale with 1 = disagree strongly and 5 = agree strongly.
Champions committee and checklist development
A committee was formed comprising representatives of all perioperative disciplines, including anesthesiology, surgery, and nursing. The committee consisted of three nurse administrators, a clinical nurse educator, eight registered nurses (RN), four surgeons (plastics, urology, orthopedics, and gynecology), three staff anesthesiologists, and an anesthesia resident. The mandate of the committee was to determine the specific items to add to the SSC that would reflect patient outcomes in an ambulatory surgical facility and to determine how the checklist would be implemented.
The Canadian version of the SSC (adapted from the World Health Organization’s Surgical Safety Checklist by the Canadian Patient Safety Institute) was downloaded from the Canadian Patient Safety Institute Web site and used as the preliminary template (Appendix A: reproduced with permission) (http://signup.patientsafetyinstitute.ca/English/toolsResources/sssl/Pages/SurgicalSafetyChecklist.aspx) This checklist consists of 59 items. “Items” were defined as any phrase preceded by either a checkbox or a dash. A literature review was then conducted to determine the most important postoperative outcomes in the ambulatory surgical population. Items related to these outcomes that had the potential to improve postoperative morbidity and could be added to the checklist were developed and presented to the committee for discussion. Items were added to the checklist if committee consensus was achieved. Seven additions were made to the WHO SSC. In the BRIEFING section: under “Surgeon(s) review(s)”, i.)intraoperative local anesthetic infiltration was added; under “Anesthesiologist(s) review(s)”, ii.)general anesthesia versus regional anesthesia,iii.)nerve block, and iv.)intraoperative antiemetics (as a single item) were added; and under “Special precautions”, v.)preemptive pain medication was added. In the TIME OUT section, vi.)allergies was added, and in the DEBRIEFING section, vii.)local anesthetic infiltration was added. The added items were specifically identified by using a red, italicized, and underlined font, and they were indicated with an asterisk. As a result, the BRIEFING section consisted of 41 items; the TIME OUT section consisted of ten items, and the DEBRIEFING section consisted of 15 items for a total of 65 items (Appendix B: reproduced with permission). In addition, a 17-item HAND OFF section was added; however, this section was not verbally included in the SSC and compliance data were not collected on this section. Rather, it was included as a guideline that the RN could use in handing over either during the procedure or in the postanesthetic care unit.
A dedicated research assistant attended surgical procedures with the principal investigator and completed the SSC data form until the principal investigator was satisfied with the accuracy of data collection. University students (n = 12) were then recruited to participate as observers and data collectors over the course of the one-year study. All students received an introduction and explanation of the study protocol by the principal investigator and research assistant. The research assistant accompanied each student in the operating room until fidelity of data collection was achieved. Students were instructed to leave the answer blank if they weren’t able to determine the answer to a question. Once completed, the research assistant reviewed all data collection forms for missing data. Chart reviews were conducted to fill in missing data points where possible.
Before introduction of the adapted SSC, trained personnel attended a random sample of ambulatory surgical procedures and prospectively collected data. Operating room personnel were not informed as to the specific reason for the observer’s presence in the operating room. All ambulatory surgical procedures were eligible for inclusion in the study. Once data collection was completed on the calculated sample size and following a multidisciplinary education session, the SSC was introduced into all operating rooms and a period of three weeks was given to allow personnel to adapt to its use. A laminated copy of the adapted SSC was posted and available in every operating room, and descriptors of the individual items were located on the reverse side of the laminated SSC. The committee decided that the surgeon would conduct the BRIEFING section, the RN, anesthesiologist, or surgeon would conduct the TIME OUT section, and the circulating RN would conduct the DEBRIEFING section using the laminated SSC as a guide.
After the introductory time period, trained observers again collected the same information as prior to SSC implementation as well as data regarding adherence to the three sections of the SSC (BRIEFING, TIME OUT, and DEBRIEFING) and to the individual items within these sections. Operating room personnel were not aware of the specific data being collected by the observers. The data collector checked off those items in each section that were verbally discussed during each surgical procedure. After all data were collected, perioperative personnel were again asked to complete the SAQ.
The primary outcome was the incidence of pain and nausea/vomiting on postoperative day one, which was obtained by telephone interview. Post-discharge pain was assessed using a scale of 1-10: 1 = no pain and 10 = the worst possible pain. Nausea and vomiting was assessed using a four-point scale: 0 = no nausea/vomiting, 1 = mild, 2 = moderate, 3 = severe nausea/vomiting. Secondary outcomes included the timing of perioperative antibiotic administration (antibiotic administration to incision time), the use of preemptive nonsteroidal anti-inflammatory medications (yes/no), the use of prophylactic antiemetics (yes/no), the use of regional anesthesia (yes/no), the use of regional nerve blocks (yes/no), the length of stay in the surgical facility, and the number of patients requiring admission.
Sample size calculation
The standard deviation (SD) of postoperative pain scores following outpatient surgery was 2.4,14 and the incidence of postoperative nausea and vomiting (PONV) ranged from 20-30%.15 Based on this information and to detect a minimum difference of one point in pain score and a 15% incidence rate, the sample size was calculated as 160 per group with an alpha = 0.025 and power = 0.80.
The data were analyzed using descriptive statistics. For continuous variables, the Shapiro-Wilk and Kolmogorov-Smirnov tests were used to test the hypothesis of normal distribution. Non-normally distributed data were reported as median (interquartile range [IQR]) and were evaluated with a Mann-Whitney U test. Categorical variables were presented as count (percentages) and were evaluated using Fisher’s exact test. For primary outcomes, P values < 0.025 were regarded as significant, corresponding to the Bonferroni correction to control the familywise error rate at 0.05 for the two tests performed. The 97.5% confidence intervals (CI) were calculated for the differences in median and percentages. For secondary outcomes, P values < 0.007 were regarded as significant, corresponding to the Bonferroni correction for the seven tests performed. The 99.3% CI of differences were calculated.
This is a 58-item questionnaire to assess staff safety attitudes, which includes an assessment of communication and collaboration among staff. A repeated measures analysis of variance was used to assess the results. Statistical significance was defined as P < 0.05.
Analyses were performed using SAS® version 9.2 (SAS Institute, Inc., Cary, NC, USA).
Pre-implementation data collection occurred over a period of seven months and post-implementation data collection occurred over a period of five months, for a total study period of one year.
Safety Attitudes Questionnaire
I would feel safe being treated here as a patient
Briefing operating room (OR) personnel before a surgical procedure is important for patient safety
I am encouraged by my colleagues to report any patient safety concerns I may have
In the ORs here, it is difficult to speak up if I perceive a problem with patient care
The physicians and nurses here work together as a well-coordinated team
Personnel frequently disregard rules or guidelines (e.g., handwashing, treatment protocols/clinical pathways, sterile field, etc.) that are established for the OR
Before implementation (n = 180)
After implementation (n = 195)
Age: mean (standard deviation)
Female: n (%)
Procedure: n (%)
Before implementation (n=180)
After implementation (n=195)
Difference (97.5% confidence interval)
Pain (POD1): median [Interquartile range] 1 = no pain; 10 = worst possible pain
0.5 (0 to 1)
Nausea and/or vomiting (POD1)
Some (mild, moderate, severe): n (%)
−8.4% (−17.9 to 1.1)
Before implementation (n = 180)
After implementation (n = 188)
Difference (99.3% confidence interval)
Antibiotic to incision time Minutes: median, [IQR]
1 (−2 to 4)
Preoperative NSAIDS given: n (%)
−23.3% (−35.5 to −11.1)
Received PONV prophylaxis, n (%)
8.0% (−3.4 to 19.4)
Regional anesthesia n (%)
−3.4% (−9.4 to 2.5)
Regional blocks: n (%)
1.0% (−11.5 to 13.0)
Length of stay in facility Hours: median, [IQR]
−0.1 (−0.4 to 0.2)
Patients admitted to hospital: n (%)
−1.1% (−4.2 to 2.1)
Items on the SSC
Compliance with the three sections on the checklist, BRIEFING, TIME OUT, and DEBRIEFING was 99.49%, 97.95%, and 96.92% respectively. Compliance was defined as a verbal discussion of at least one item in the section. The median [IQR] number of items verbalized was: 19 [15-22] items in the 41-item BRIEFING section, 5 [4-6] items in the ten-item TIME OUT section, and 9 [7-11] items in the 15-item DEBRIEFING section.
The purpose of this study was to adapt an already developed checklist to include items relevant to the ambulatory surgical population. Ambulatory surgery has low rates of major morbidity and mortality so outcomes, such as postoperative pain and PONV or post-discharge nausea and vomiting (PDNV), are more relevant in this population.17 There is evidence to suggest that attention to certain measures can significantly decrease the severity of postoperative pain and PONV/PDNV, including such interventions as preemptive pain medications, the use of regional anesthesia or regional nerve blocks, and the use of prophylactic antiemetics, especially in high-risk surgical procedures/patients.15 Despite adding items to the SSC that pertain to these specific interventions, verbalization of these items occurred in < 10% of observed cases. Consequently, it is not surprising that there was no improvement in post-discharge pain and nausea/vomiting on the first postoperative day before and after checklist introduction. We also did not show a difference in the use of regional anesthesia or prophylactic antiemetics. There was a significant increase in the use of preemptive analgesics after SSC introduction. This difference resulted from a change in process whereby orthopedic patients began to receive preoperative nonsteroidal medications on a routine basis during this period of time; this change was unrelated to the introduction of the SSC.
Checklist use has been a mainstay of safety practices in the aviation industry for years. The use of checklists in health care, however, has been much slower to evolve despite evidence that their use improves patient outcomes and/or improves team behaviours and communication.5,18-23 Specifically, use of the WHO SSC has shown a decrease in inpatient morbidity and mortality2,4-7,24 as well as improvement in teamwork skills, communication, and adherence to process measures.1,23,25-27 Nevertheless, Ko et al.’s recent systematic review of checklist use in medicine has shown limited evidence of their effectiveness.28 Analysis of the studies indicated that many were of poor quality with low levels of evidence and a high rate of bias. In summary, the authors determined that it was not feasible to provide an accurate summary of any trends across all studies.
Similar to the study by Vogts and Fourcade, compliance with BRIEFING and TIME OUT was high in our study, but our study also showed a high compliance with the DEBRIEFING section.29,30 The three items most commonly discussed were: Allergies, Site and procedure, and ASA Class (American Society of Anesthesiologists’ class). There is, however, a need for caution when interpreting compliance rates. Compliance is often defined as some discussion having occurred with respect to the three sections of the SSC; however, not all items may have been addressed within each section. Levy et al. recently reported that individual items were either not executed as designed or not executed at all despite the documented 100% completion of the SSC BRIEFING section.31 Therefore, some studies report on “completeness” of the SSC. The evidence suggests that rate of “completeness” of the SSC is significantly lower than the rate of compliance.5,30,32 Our study has shown that 21 items were discussed < 33% of the time.
There are a number of reports of low compliance with healthcare checklists.33,34 In this study, several factors may have contributed to the low compliance within certain areas of the SSC and the failure to use the checklist to improve the primary and secondary outcomes. For a checklist to be effective, users at the “sharp end” must appreciate its value.35 If the users are told simply that the checklist is an institutional requirement and they are not given or are not interested in the rationale for its implementation, adherence to the checklist may be suboptimal. Certainly, in the case of the SSC, its implementation was an institutional requirement and an institution’s adherence to the SSC was posted on a provincial Web site with public access. Although attempts were made to inform all stakeholders of the process and the rationale behind the use of the SSC, the lack of improvement suggests that the SSC was not implemented effectively.
This particular point is reinforced by our results from testing safety attitudes. Safety culture has been recognized as an important strategy in the improvement of the deficits identified in healthcare safety.36 Culture has been identified as a factor affecting absenteeism, employee turnover, length of stay, and readmission rates. The SAQ is a validated tool designed specifically for operating room teams and is an adaptation of the Flight Management Attitudes Questionnaire developed for aviation teams.37 Using six items from the SAQ, Haynes showed an improvement in safety attitudes before and after introduction of the SSC.38 Our study did not show an improvement in safety attitudes using these same six items, which may be accounted for by the higher values seen in our pre-intervention group (Table 1).
The checklist may have been too lengthy for use in an ambulatory surgical facility where turnover pressure is high. Limiting the length of a checklist is considered crucial for its feasibility and usefulness. In the nuclear industry, it has been stated that “…as the list of items grows, there may be a higher probability of overlooking any given item.”39 Had we pilot tested the initial adaptation of the checklist at an early stage, we may have been able to identify the checklist items that were not consistently noted, allowing for an initial modification and obviating the need for a revised checklist at the end of the study.
The Pareto principle may be employed with respect to checklist content. It assumes that 20% of adverse events will account for 80% of injuries, allowing tailoring of a checklist to focus on priority areas. If potential adverse events are not prioritized, healthcare workers may end up addressing 80% of the adverse events that account for only 20% of the injuries.35 In this study, addition of more items to an already lengthy checklist was likely compounding the lack of prioritization of SSC items. Although morbidity from post-discharge pain and nausea/vomiting occurs, major adverse events more likely to profit from a “checklist” discussion would include four items: correct patient, correct operation, correct side, and allergy identification. In fact, despite the length of the SSC used in this study, these four “big ticket” items were identified in > 90% of cases. All operating room staff may have clearly understood the rationale for discussion of these items and were therefore compliant. Other items, however, may not have been perceived as relevant by all users and therefore were not mentioned. Similar to the comments of Vats et al., some items on the SSC were simply not relevant to cases in an ambulatory facility and were therefore ignored.33
Another potential confounding factor in the lack of effective SSC implementation was the fact that the surgeons involved in the study were not all based primarily at our centre. They would therefore have been exposed to a different checklist at another institution and may not have recognized the differences between their home-based hospital SSC and our ambulatory facility SSC. Hierarchy may also have played a role in that some RNs may have been reluctant to “step in” if items were not mentioned by the attending surgeon.
In summary, the SSC was not effectively implemented in our ambulatory facility. A number of potential reasons have been cited. Specifically, items added to the SSC in an attempt to improve the primary outcomes of post-discharge pain and nausea/vomiting were ineffective, suggesting that the SSC may not be the optimal strategy to address these issues. Due to poor compliance with the adapted version (Appendix B) used in this study, a final abbreviated checklist has been developed for use in an ambulatory surgical facility and is available from the authors.
This study was funded by a grant from the Alternate Funding Plan Innovation Fund, Ministry of Health and Long Term Care and Department of Anesthesia, University of Toronto Merit Award.