Electronic Voting to Improve Morbidity and Mortality Conferences
It is of major importance in clinical surgery to identify potential patterns and specific causes of complications. Therefore, morbidity and mortality meetings (M&M) are widely used to discuss and evaluate deviations from expected outcomes in order to improve surgical practice. Moreover, M&M represent an important tool for continuous medical education. In this study, we introduced an electronic voting system to assess whether anonymity during M&M could limit potential biases due to hierarchical structures or opinion leaders.
This study was conducted in the surgical department of a European tertiary care center. During the study period, electronic voting was applied in 412 M&M cases and compared with a baseline of 330 conventional M&M entries. In this interrupted time series, the educational quality and participant satisfaction of the M&M were assessed using surveys before and after the introduction of electronic voting. The surveys were refined using principle component analysis. In addition, the classification of the cause of the complication was recorded.
The introduction of electronic voting led to a significant increase in perceived educational quality from 2.63 to 3.36 (p < 0.01), and the overall participant satisfaction increased from 2.6 ± 0.9 to 3.7 ± 1.2 (p < 0.01) on a five-point Likert scale. The frequency of voting shifted from “patient’s disease” (before 42.9, after 27.6%, p = 0.04) to “misadventure” (before 1.1, after 16.0%, p < 0.01). The voting frequencies for the causes attributed to “management” and “technical” remained constant.
An electronic voting system in M&M meetings increases perceived educational quality and participant satisfaction.
Morbidity and mortality
At least half of all surgical complications are avoidable [1, 2, 3]. Morbidity and mortality meetings (M&M) provide a means to identify avoidable complications and thereby to improve surgical and medical management [4, 5, 6, 7]. The main goal of M&M is to analyze medical incidences in order to better understand causative factors and to assess alternative decision-making. In parallel, M&M ensure continuous medical education of trainees and staff [8, 9, 10, 11].
For a comprehensive discussion of medical errors, it is of utmost importance to ensure standardization of M&M especially within a department and ideally between institutions. Attempts to standardized M&M include root-cause analysis (RCA) which is an approach to identify the underlying cause or causes of a problem. It is designed to answer three basic questions: what happened, why did it happen, and what can be done to prevent it from happening again . Another attempt to generate standardized safety communications in M&Ms is the SBAR (Situation, Background, Assessment, and Recommendations) which was originally introduced in high-risk industries from where it was adopted for use in medicine . However, the adoption of RCA and SBAR in medicine, and in particular surgery, is hampered by specific limitations such as underreporting [14, 15, 16], lack of sufficient information, inadequate presentation [17, 18], and especially by the potential for differences due to hierarchical structures [17, 18, 19, 20].
Standardized identification, reporting, and presentation of cases. Self-reporting of cases by individual surgeons should be circumvented as this approach is prone to underreporting: surgeons may avoid discussions of complications of their own patients [14, 19, 20]. Standardized methods to identify adverse events and preventable deaths have been described in the literature. Various systems have been implemented in the surgical environment such as the global trigger tool , or reporting systems that include external validation, such as the national American College of Surgeons-National Surgical Quality Improvement program (ACS-NSQIP). Trigger tools, registries, or other quality assurance initiatives, have been shown to significantly increase the rate of complications detected [13, 14, 15, 16, 21].
Objective and standardized analysis and discussion of complications. The discussion should classify complications into causative categories. Despite the lack of a consensus on a classification, the distinction between preventable and unpreventable is widely accepted [4, 8, 22, 23, 24, 25, 26]. Preventable causes include technical aspects, decision-making, and team factors. Potentially unpreventable causes include patient-related factors, simple bad luck and unknown reasons for a complication [16, 27]. However, such classifications do not reflect that the majority of medical errors are based on multifactorial causes . To classify complications, an open and free discussion, that allows all participants to contribute, is important. However, clinical or hierarchical opinion leaders may bias such an open discussion [4, 19, 25, 29, 30, 31].
Translation: Implicit or explicit translation of what was learned into daily clinical practice should be attempted by formulating general or specific recommendations [10, 13]. This element is reflected by the major role M&M play in educational initiatives. In the USA, M&M have become a required part of training for both surgical and medical residents and many countries, including the UK, have followed suit [7, 32].
The aim of this initiative was to improve discussion, satisfaction, and education for participants attending M&M. Therefore, an electronic voting system was introduced to encourage free decision-making and to allow the simultaneous identification and assessment of the relative importance of multiple causes of complications. Electronic voting was embedded into a novel framework that was incorporated into the existing SBAR framework. It includes standardized case selection, standardized case presentation and whenever possible a recommendation.
Within the present prospective cohort study, we explored the impact of electronic voting on participant satisfaction and educational quality.
Materials and methods
This study was performed as an interrupted time series design in a European tertiary care center. Baseline measurements were assessed before and compared to measurements after the introduction of electronic voting.
Before the start of the study, identification and selection of patients was based on a voluntary reporting system. At M&M, case presentation was done by an intern familiar with the case, followed by a free discussion, led by a senior consultant. After the discussion, the complications were categorized according to the usual classification of the institution into the categories “technical,” “management,” “patient’s disease,” or “misadventure.” Votes for classification of complications were recorded electronically in a database. During the baseline period, unsystematic iterative observations by professional work psychologists were performed in order to identify potential bias. These observations were the basis for the design of the novel framework that includes electronic voting.
The novel framework and electronic voting was introduced on December 5, 2014. After a test period of 4 months, the electronic voting was definitively installed on April 1, 2015, and was formally validated within the current study.
Standardized identification and selection of cases was implemented based on an existing quality assurance initiative. All surgical procedures were continuously listed by external study nurses. At least 30 days after surgery, all patient records were screened for any deterioration of normal recovery by an independent (i.e., non-treating) physician. This physician did not receive any specific training. The selection process was evaluated and certified within a process audit (Deutsche Krebsgesellschaft e.V. OnkoZert, ISO 9001, registry number: FAD-Z333 V). All deteriorations were graded according to Clavien–Dindo Complication Score  and entered into a quality control database. All complications scoring three or higher were selected to be discussed at the next M&M. Lower graded complications were discussed only upon request from the responsible surgeon.
A specific graphical representation was used in order to clearly define and visualize the entire workflow (Fig. 1). Structured case presentation was done by the treating physician using a standardized Microsoft® PowerPoint™ 2010 template as previously described . All presenting physicians were trained in the novel workflow for 1 h by the investigators. The presentations were delivered by the lowest ranking intern of the treating team.
Electronic voting and assessment
Technical Attributed to the application of improper surgical technique.
Management Occurred due to potential errors in the perioperative management of the patient.
Patient’s disease Due to an underlying disease of the patient. Failures in management should be ruled out.
Misadventure Due to a random event that cannot be possibly controlled for by proper technique and/or management.
Unclear Information available is precise but ambiguous.
Not enough information Information available is not precise enough/vague. Further information should be obtained whenever possible.
The voting results were displayed as percentages for each possible answer on-screen using Microsoft® PowerPoint™ 2010.
According to the framework (Fig. 1) “alternative treatment options” were considered. If the evidence present was sufficient, a recommendation was directly formulated. If not, a short review of the literature and a choice of recommendations were prepared for the next M&M.
All patients operated between January 1, 2013, and December 31, 2016, were included. Number of operations performed per year and number of identified complications were registered. Patient data were extracted retrospectively after anonymization from the M&M database.
Number of participants
Position n (%)
Surgical experience in years (±SDa)*
Gender female (%)
All non-demographic items were answered on a five-point Likert scale with 1 being the lowest and 5 being the highest value. The survey was performed before and after the introduction of the novel framework. The first survey took place between September 1 and November 30, 2014. The second survey was conducted between February 22 and March 31, 2015 after training of the staff in the novel framework. The anonymous, electronic survey was sent to all physicians and students participating in the M&M. Primary non-responders were reminded by additional emails and direct contact.
To be able to compare surveys of the same participants before and after the intervention, participants entered a confidential code, used to match the surveys. Independent-sample test between participants answering only one survey revealed no significant difference (p = 0.88). Only participants who completed both surveys were included in the statistical analysis.
Voting results and complications of M&Ms have been prospectively entered since January 2010 within a quality control database. The voting results of the last year before the introduction of electronic voting (January 1, 2013, until December 31, 2013) were compared to the first year following the introduction of electronic voting (January 1, 2016, until December 31, 2016).
Statistical analysis was performed using SPSS (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.). P values were two-tailed, with p < 0.05 determining statistical significance. Except for demographic data, all questionnaire items were based on a five-point Likert scale. Data are presented as mean (±standard deviation). Statistical significance of principle components (five-point Likert scale data) was assessed using paired t tests . Kruskal–Wallis H tests were used to analyze the data from the voting results before and after the introduction of the M&M framework.
The final response rate for each survey was 87% (40/46) and 94% (33/35) (Table 1).
During the study period electronic voting was applied in 1147 M&M cases. As a baseline comparison, 330 entries of conventionally held M&M entries were extracted.
Institutional error culture
Institutional error culture (PC1) remained constant at 3.0 (±0.59) before and 3.1 (±0.73; p = 0.51) after the implementation of electronic voting (Fig. 2a). The baseline error culture of the department is shown in Supplementary Fig. 1.
Goals and consequences of M&M
All principle components of the survey category “goals and consequences of M&M” were significantly increased (Fig. 2b): PC1 assessed the relevance of the in-depth discussion and increased from 1.78 (±0.38) to 1.97 (±0.49) (p = 0.045). PC2 assessed the ability of the M&M to discriminate personal from institutional problems and increased from 2.73 (±0.50) to 3.95 (±0.76) (p < 0.01). PC3 assessed the implementation of recommendations into daily practice and increased from 3.00 (±0.55) to 3.48 (±0.81) (p = 0.04).
Individual perceived benefits of M&M
After introduction of electronic voting, the perceived personal benefits of M&M increased significantly (Fig. 2c). PC1, which assessed the integration of delicate topics, was not significantly different (before 3.33 (±0.85), after 3.56 (±0.75), p = 0.33). PC2 assessed psychological safety (e.g., “I feel comfortable during M&M”) and increased significantly from 2.99 (±0.74) to 3.44 (±0.80) (p = 0.02). PC3 assessed the educational effectiveness and increased significantly from 2.63 (±0.71) to 3.36 (±0.88) (p < 0.01).
Overall satisfaction increased from 2.6 to 3.7 (p < 0.01) (Fig. 2d).
Votes for causes of complications
M&M are a valuable and simple tool for quality control and education. In this study, we have designed and implemented an electronic voting system within a novel M&M framework. We observed a significant increase in the perceived benefit of M&M by the participants, particularly with respect to its educational effectiveness. The benefit of this novel framework may be a consequence of electronic voting and its effect in reducing hierarchical bias therefore empowering the whole audience. In the proposed SPEAR framework, electronic voting was performed after the case was discussed, therefore a remaining influence of opinion leaders cannot be ruled out. In future validation studies of this framework, voting could take place before the discussion of the case and after to infer on the influence by senior/opinion leader staff.
In addition, overall satisfaction “goal and consequences of M&M” were significantly increased, indicating an increase in validity and acceptance of the M&M’s conclusions.
The selection of complications has been highly standardized by the introduction of the novel M&M framework and electronic voting. The introduction of electronic voting was associated with an increase in the scope of the interpretation of complications. The category “misadventure” was chosen more frequently potentially revealing an uncertainty of the causes. This supports the hypothesis of a potential bias because of hierarchical structures before the introduction of electronic voting in which judgment was more dependent on the opinion of the leader of the discussion. However, the number of complications judged to be potentially preventable, such as “technical” or “management” errors, remained constant at around 50%. This is in line with the often-cited estimation that half of medical complications are preventable [1, 2, 3].
No change in institutional error culture was observed, this being potentially based on a well-accepted error culture within our department at the beginning of this study. Thus, the current study reveals that improvement of M&M is possible by providing key structures without fundamental change in the underlying error culture. Electronic voting seems to be the most important reason for the results of this study. In this study a voting tool was used that needs the purchase of additional hardware (iml click®, Lumi insight). The accompanying software ViewPoint is free. As simple and effective alternative tools, existing smartphone applications could be used. Several Web-based smartphone applications are provided free of charge and have been validated for academic use . Another interesting opportunity may lie in a combination of electronic voting along with interactive multi-site video teleconference M&M as recently proposed .
The introduction of electronic voting was associated with a significant increase in the educational quality of M&M.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Compliance with ethical standards
Conflict of interest
The all authors declare no conflict of interest with this paper.
- 9.Deis JN, Smith KM, Warren MD et al (2008) Transforming the morbidity and mortality conference into an instrument for systemwide improvement. In: Henriksen K, Battles JB, Keyes MA, Grady ML (eds) Advances in patient safety: new directions and alternative approaches, vol 2: Culture and redesign. Agency for Healthcare Research and Quality, Rockville, MDGoogle Scholar
- 10.de Vos MS, Hamming JF, Marang-van de Mheen PJ (2017) Learning from morbidity and mortality conferences: focus and sustainability of lessons for patient care. J Patient Saf. https://doi.org/10.1097/PTS.0000000000000440
- 11.Singh HP, Durani P, Dias JJ (2015) Enhanced morbidity and mortality meeting and patient safety education for specialty trainees. J Patient Saf. https://doi.org/10.1097/PTS.0000000000000208
- 16.Hutter MM, Rowell KS, Devaney LA et al (2006) Identification of surgical complications and deaths: an assessment of the traditional surgical morbidity and mortality conference compared with the American College of Surgeons-National Surgical Quality Improvement Program. J Am Coll Surg 203(5):618–624CrossRefGoogle Scholar
- 22.Becker A (2013) Quality criteria for successful morbidity and mortality conferences (Qualitätskriterien erfolgreicher Morbiditäts-und Mortalitätskonferenzen). Interdisciplinary Contributions to Hospital Management: Medicine, Patient Safety and Economics (Internet). 10 Dec 2013; (Paper ID #015). http://www.clinotel-journal.de/article-id-015.html
- 23.Dijkema LM, Dieperink W, van Meurs M et al (2012) Preventable mortality evaluation in the ICU. Crit Care 16(309):12Google Scholar
- 26.Travaglia J, Debono D (2009) Mortality and morbidity reviews: a comprehensive review of the literature. Centre for Clinical Governance Research, University of New South WalesGoogle Scholar
- 35.Vellis D (2012) scale development: theory and applications, 3rd edn. Sage, Los AngelesGoogle Scholar
- 36.Winter JCFd, Dodou D (2010) Five-point Likert items: t test versus Mann–Whitney–Wilcoxon practical assessment. Res Eval 15(11):1–16Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.